keyValueStore / JSON Store Plugin

What are KeyValueStore / JSONStore Plugin?

The KeyValue Store Plugin is an easy way to save a certain structure of data with a unique identifier = key and value

For Example

Key	Value
Name	Christian

Important Update regarding Persistence Layer

Up to Simplifier 7.0, the plugins JSONStore and KeyValueStore are using MapDB as storage Backend by default. MapDB writes all entries into one large file.

Beginning with 6.5.151 and 7.0.44, a config setting was introduced wich gave the possibility to use the default Simplifier Database as backend for these to plugins.

With Simplifier 7 EHP 1, this setting is removed and the default database is the only supported backend for JSONStore and KeyValueStore Plugins.

The documentation describes 2 scenarios. You need to follow only one of them. Follow Scenario 1 if you have or are close to a default setup. This means, you use MySQL as database backend and you want to have the target for the migration in the same database server as the main Simplifer database is located. If you have different needs, please follow scenario 2. With that, you are able to configure the backend databases very detailed.

Backup your database files

Before starting the migration, make sure you have a backup of the two database files. The files to backup are:

${SIMPLIFIER_DATA_DIR}/plugins/jsonStore/jsonStore
${SIMPLIFIER_DATA_DIR}/plugins/keyValueStore/kvstore

In a default setup using our docker compose file, ${SIMPLIFIER_DATA_DIR} is /var/lib/simplifier/data.

Scenario 1: Default Setup

Follow either scenario 1 or scenario, never both.

This is the standard setup which most people should follow.

Before you do the Simplifier update: Create Databases

Create two new database in your default Database server (the one currently used for your simplifier database).

If you use the recommended docker compose setup including the MySQL Database Server container image:

Connect to the host via SSH

You need the following informtion from your .env file:

MySQL root user password: DB_ROOT_PASSWORD
Database base name: DB_NAME

Connect to your MySQL server

docker exec -it mysql /bin/sh
mysql -u root -p

Inside the MySQL console, run the following commands. Substitude ${DB_NAME} with the value of DB_NAME from .env file.

CREATE DATABASE ${DB_NAME}_keyvalue;
GRANT ALL PRIVILEGES ON `${DB_NAME}_keyvalue`.* TO `simplifier`@`%`; 
CREATE DATABASE ${DB_NAME}_jsonstore;
GRANT ALL PRIVILEGES ON `${DB_NAME}_jsonstore`.* TO `simplifier`@`%`;
FLUSH PRIVILEGES;

Scenario 2: Individual Setup

Follow either scenario 1 or scenario, never both.

This scenario is relevant if you want to control the setup in more details.

Before you do the Simplifier update: Create Databases

Create two new databases, one for the Key-Value Store and one for the JSON Store. If necessary, create a user or grant an existing user all access rights to these 2 new databases.

For example, if you already have an DB user called sf_db_user and if you are using a MySQL database server:

CREATE DATABASE simplifier_keyvalue;
GRANT ALL PRIVILEGES ON `simplifier_keyvalue`.* TO `sf_db_user`@`%`; 
CREATE DATABASE simplifier_jsonstore;
GRANT ALL PRIVILEGES ON `simplifier_jsonstore`.* TO `sf_db_user`@`%`;
FLUSH PRIVILEGES;

Create Configuration

Go to the Simplifier data directory. Then

cd conf

In this directory, you need to create two new files, one for Key-Value Store and one for JSON Store plugin. Use your favourite editor, e.g. nano. These files have to be named exactly like this.

nano jsonStore_include.conf
nano keyValueStorePlugin_include.conf

In this file, write the folling content if you have a MySQL server. Substitute appropriate values for user, password, database host and database name. The content for both plugins is similar, just write the appropriate database name.

database {
  dbms: "mysql"
  user: "YourDatabaseUserName"
  pass: "YourDatabasePassword"
  host: "HostnameOfYourDatabaseServer"
  port: 3306
  database: "NameOfTheDatabaseCreatedBefore"
  table_prefix: ""
}

If you have an Oracle Database server:

database {
  dbms: "oracle"
  user: "YourDatabaseUserName"
  pass: "YourDatabasePassword"
  host: "HostnameOfYourDatabaseServer"
  port: 1527
  database: "NameOfTheDatabaseCreatedBefore"
  table_prefix: ""
}

After the first Start of Simplifier after updating to 7.1, a migration is started to move all data from the old Backend to the new Backend. Depending on the size of data file, this migration takes time and needs a performant database backend (20 GB Data requires a MySQL Server with 16 GB RAM temporary). If you have only a very limited server, consider increasing its ressourcing during this migration phase. Afterwards you can lower ressources back to your normal settings. Migrating a larger dataset may take one hour and more.