Data Migration Guide

This article provides Iotellect-specific guidance for transitioning platform storage from Cassandra 3.x to Cassandra 5.x during upgrades from Iotellect 6.3 to Iotellect 6.4 (or later). It covers the following scenarios:

• Iotellect 6.3 → 6.4 (or later) on Linux using embedded Cassandra (automatic upgrade: 3.x → 5.x).

• Iotellect on Windows upgrading to 6.4 (or later) when embedded Cassandra is used (requires migration to external Cassandra).

• Iotellect installations connected to an external Cassandra deployment (standalone or cluster) where only Iotellect is upgraded (Cassandra remains managed separately).

• The procedures and checks can be applied to development, staging, and production environments (adjust your maintenance window and rollback plan accordingly).

Apache Cassandra does not support in-place upgrades between major versions 3 and 5.

Specifically:

• SSTable formats are incompatible

• Storage engine internals have changed

• Metadata and system keyspaces differ

• Compaction and indexing behavior has evolved

Attempting to reuse Cassandra 3 data files with Cassandra 5 will result in startup failure or silent data corruption.

In this guide, the recommended path depends on your deployment model.

Iotellect uses Cassandra as a core persistence layer for:

• Historical variable values

• Event history and acknowledgements

• Statistics and analytical aggregates

• Internal system metadata required for platform operation

Because of this:

• Partial migrations are not supported

• Schema integrity is critical

• Data loss may cause undefined platform behavior.

Cassandra 5 cannot run in embedded mode on Windows. Any Windows-based installation must migrate to external Cassandra 5.

All manual migration scenarios follow the same lifecycle:

1. Controlled shutdown of Iotellect Server

2. Snapshot creation on Cassandra 3

3. Deployment of Cassandra 5

4. Schema-compatible data import

5. Iotellect Server configuration validation

6. Functional and data integrity verification

Each step is mandatory and must be completed sequentially.

Before starting migration, ensure all conditions below are met.

• Full filesystem access to Iotellect directories

• Permission to stop/start services

• Cassandra administrative privileges

Create independent, restorable backups of:

• Cassandra 3 data directory

• Iotellect Server configuration (conf, config, or equivalent)

• Custom drivers, plugins, and extensions

• SSL certificates and keystores (if used)

Migration without verified backups is unsupported and unsafe.

Ensure that:

• The target Iotellect version explicitly supports Cassandra 5

• No third-party extensions rely on Cassandra 3-specific behavior

For installations that use an external Cassandra deployment, there is no Cassandra 3.x → 5.x data migration as part of the Iotellect 6.3 → 6.4 upgrade. The external Cassandra servers remain managed and upgraded separately.

For Linux systems using embedded Cassandra, the Cassandra engine upgrade is performed automatically as part of the Iotellect upgrade (6.3 → 6.4 or later).

• Back up the Iotellect installation directory at <AG_HOME> and any external backups required by your operational policy.

• Upgrade Iotellect from 6.3 to 6.4 (or later) using the standard upgrade procedure.

• Start Iotellect Server and confirm that embedded Cassandra starts successfully (check server logs) and that historical data and event storage are accessible.

• Complete the validation checklist.

Use the manual snapshot-based migration only if automatic upgrade cannot be done.

systemctl stop aggregate

Confirm that:

• No Iotellect Server Java processes remain

• Cassandra remains reachable for snapshot creation

nodetool snapshot requires a running Cassandra process. Do not stop Cassandra before completing Step 2.

Run on the Cassandra 3 source node (terminal/command line) using the Cassandra 3 nodetool, then execute:

nodetool snapshot

This operation:

• Creates a consistent, point-in-time snapshot

• Preserves data across all keyspaces

• Does not interrupt Cassandra operation

Locate snapshot directories, typically under:

<AG_HOME>/data/cassandra/data

Copy the entire directory tree to secure storage.

After snapshot data is preserved, ensure the Cassandra 3 process is no longer running before performing the software upgrade in Step 4 (to avoid conflicts during upgrade).

Install the Iotellect version that includes embedded Cassandra 5.

Do not reuse old Cassandra directories.

Start Iotellect Server once to allow Cassandra 5 to:

• Create system keyspaces

• Initialize schema metadata

• Prepare directory structure

Then stop Iotellect Server again.

Before loading data (Step 6), ensure the target Cassandra 5 schema is initialized and compatible with the incoming data.

Using Cassandra 5 tooling execute:

sstableloader <snapshot_directory>

Key notes:

• Use the sstableloader tool from the Cassandra 5 distribution

• Load one keyspace at a time if required

• Monitor logs for schema or compatibility warnings

systemctl start aggregate

Proceed to validation.

This scenario is required for Windows installations.

Stop the Windows service or application instance.

Run on the embedded Cassandra 3 instance (terminal/command line) using a matching Cassandra 3 nodetool:

nodetool snapshot

Snapshot location (Windows):

<AG_HOME>\data\cassandra\data

Copy snapshot directories from:

<AG_HOME>\data\cassandra\data

Copy the snapshot directory tree to secure storage.

Install Cassandra 5 on a supported host.

Ensure:

• Stable network connectivity

• Correct listen and broadcast addresses

• Sufficient disk and memory resources

Before import, ensure the Cassandra 5 schema is initialized and compatible with the incoming data (e.g., by starting the target Iotellect Server version once against the external Cassandra, or by applying the required schema through your standard process).

Run on Cassandra 5 host:

sstableloader <snapshot_directory>

Update Iotellect Server configuration to point to the external Cassandra instance:

• Hosts

• Ports

• Authentication (if enabled)

• Data center settings

Proceed to validation.

• Cassandra reports UN (Up/Normal) state

• No schema mismatch errors in logs

• Stable connection latency

Verify:

• Iotellect Server starts without database warnings

• Context tree loads fully

• Dashboards render correctly

• Event history is accessible

• Historical data queries return expected results

Test:

• Variable updates and history

• Event generation and acknowledgment

• Alerts and rules

• Reporting and analytics

If migration fails:

1. Stop Iotellect Server

2. Shut down Cassandra 5

3. Restore Cassandra 3 data from backup

4. Restore original Iotellect version

5. Restart services

Rollback is only possible if no data was written to the new Cassandra 5 instance after migration.

• Large datasets may require extended load times

• Snapshot loading is CPU and disk intensive

• Clustered environments may require node-by-node loading

• Post-migration compaction may run automatically

• Perform the migration during a controlled maintenance window

• Ensure data at rest encryption compatibility

• Validate access control after migration

• Preserve the audit logs across migration

• Working with Embedded Cassandra Database

• Working with External Cassandra Database

• Failover Cluster

• Migration Guide

• Cassandra 3 to Cassandra 5 migration requires explicit data transfer for migrations between separate Cassandra installations.
  For Linux deployments using embedded Cassandra, the 3.x to 5.x transition is performed automatically during Iotellect 6.3 to 6.4 upgrades; perform backups and validation.

• Embedded Cassandra 5 is Linux-only.

• Windows systems must use external Cassandra.

• For manual migrations between separate Cassandra installations, snapshot-based migration is the supported method.

• Proper validation is essential for platform stability.

Was this page helpful?

Tell us more

Send

Thank you!