Content Migration

 

The format of the AppData changes between versions. Certain types of content may be moved to a different location with the AppData e.g., bookmarks from loose files into the repository for version tracking and cluster replication. Frequently, new features may be added to the content which may include changes to the content models, e.g., workbooks with new property. Typically a newer server version will do this migration behind the scenes when it first starts up, and any issues found will be logged with at least a warning level.

 

NOTE	After starting a new server version for the first time, check the log file for warnings. It is much easier to address these issues immediately than later on, for example, if you wish to redo a one-time migration step.

 

By default, the server will check for content in an old location in AppData and do a one-time migration of anything it finds when it starts. Typically, the server will copy old content from loose files into the repository. This type of migration is controlled through the repository.migrate.<type>.path properties in Panopticon.properties.

For example, in versions 21.1 and older, the server stored workbook bookmarks as loose files in the AppData/Bookmarks/ directory. Version 21.2 stores them inside the repository in AppData /.repository/ instead. The property repository.migrate.bookmarks.path defaults to Bookmarks which is relative to AppData and therefore points to the 21.1 bookmark files. When the 21.2 server starts, this can happen:

q  There are no bookmarks in the repository, but are available in AppData/Bookmarks/ or to some other location you have set the property to. The server will do a one-time migration and move them into the repository. The result of this will be logged. The old bookmark files are left in the old location but will no longer be used.

q  There are bookmarks in the repository. You may have created them on the new server, or the migration has run already. In this case, it does not matter if the old location has bookmarks or not, and the server will log a warning that it will not run a second migration. To get rid of the warning, simply blank out the property value.

q  There are no bookmarks in either location, but the property is still set. This would be the default on a new server. In this case, you will get a false migration warning because the server cannot find AppData/Bookmarks/. Again, just blank out the property value to get rid of the warning.

 

NOTE

·         Because the server will not migrate a type of content (e.g., bookmarks) if that type of content is already in the repository, you will need to delete the repository to run the migration again. The easiest way is to start over with the AppData from the old server. ·         After the content has been migrated, the original files are left in the old location in AppData. They are no longer used, to clean up, you may want to delete them after you have checked the logs for any migration issues.

 

There are some types of content that have changed so much between releases that they cannot be automatically migrated like this.

Workbooks and their history in version 17 and older were stored separately in AppData/Workbooks/and AppData/Archive/. You can use the repository.migrate.workbooks.path and repository.migrate.archive.path properties to migrate them, but we do not default these to the old locations (e.g., you may not want to migrate the entire history), and for clarity you need to use absolute paths if you set them.

Data files used with text data sources can now optionally be stored in the repository so they can be bundled with the workbook, and also replicated to other servers in a cluster. You can still have data files in AppData/Data/, so old workbooks will continue to work on new servers, but old files are not automatically migrated into the repository.

Permissions on workbook folders were in version 17 and older stored in XML files in the AppData/Workbooks/ subtree. The permissions model has changed completely since then, so they are not automatically migrated. To migrate permissions from version 17, you need to:

1.      Use PCLI convertpermissions to create a permissions template which, as closely as possible, reflects the old permissions. This is a single JSON file which the new server can apply to its repository.

2.      Review the generated permissions template in a text editor to make sure it is correct.

3.      Point the repository.startup.apply.permissions.path to the template file and start the server. You can control how the template is applied with the properties repository.startup.apply.permissions.clean and repository.startup.apply.permissions.create.

4.      Clear the properties after the server has started, or they will be applied on each startup overwriting changes you make.

 

NOTE	In version 17 and earlier, the files AppData/Administrators.txt and AppData/AdministratorsGroup.txt were used to give users administrative permissions. Currently, with the normal permissions model, these files are no longer used.

 

Authentication tokens are server-specific. They will only work on a new server if it has the same authentication.token.secret value as the old server that created them. In addition, a normal user token is also stored as cookie in the user's browser, and will only get sent to a new server if it has the same URL as the old server. For these reasons, tokens are not automatically migrated  and users will have to log in again.

The exception to token migration is API tokens. In version 21.1 and older, these were stored in AppData/APIToken/. In 21.2, all tokens, including the API tokens, are stored in the shared cluster storage (even if you only have one single server), by default in AppData/shared/tokens/. See also the section on Synchronizing Tokens. If the server finds API tokens in the old location, they will be migrated on startup.