Using Historian

In classic SCADA systems, the historian is a critical component responsible for accumulating, storing, and retrieving historical data. This historical data, which can comprise time-series values from devices, alarms, events, and other system logs, is essential for a variety of tasks, including:

  • Trend Analysis: Visualizing historical data trends to identify patterns and anomalies.

  • Reporting: Generating reports based on historical data for regulatory compliance, performance analysis, or other purposes.

  • Troubleshooting: Investigating past events to diagnose and resolve system issues.

  • Optimization: Analyzing historical data to optimize processes and improve efficiency.

Iotellect offers a comprehensive set of features that cover and extend the core functionality of a traditional SCADA historian with advanced data processing, flexible aggregation methods, and seamless integration with other platform modules. The following sections detail how Iotellect addresses the key aspects of a historian.

Data Acquisition and Storage

Iotellect acquires historical data from multiple sources:

Data Aggregation

While raw historical data is essential for detailed analysis, aggregated values often provide a more manageable perspective for identifying trends and anomalies. Iotellect provides two flexible aggregation methods:

  • Round-Robin Database (RRD): The embedded RRD aggregates numeric data over fixed time periods (minutely, hourly, daily, etc.), calculating averages, minimums, maximums, totals, first, and last values. RRD offers high-performance data updates and retrieval, making it suitable for real-time trending and charting. The Statistics function provides access to aggregated data in the RRD, while the Raw Statistics function retrieves the complete raw dataset collected by a statistical channel. Configuration of statistics channels, including channel type (gauge, counter, derive, absolute), aggregation functions, and storage periods, is detailed in the Statistics section.

  • Granulation: The Granulation Engine provides flexible aggregation over custom, potentially unevenly-spaced time periods. It supports accumulating any data type, not just numbers, enabling complex aggregations such as concatenating strings or merging data tables. Data is accumulated in granules, which are special events whose data tables hold the aggregated values. Granules can be accessed using the Get Event History function. The granule configuration, including value format, expressions for calculating aggregate values and timestamps, and expiration periods, is detailed in the Granulation section.

Data Retrieval

Iotellect offers unified methods for retrieving both raw and aggregated historical data, irrespective of the underlying storage facility. Specialized functions facilitate efficient data loading:

  • Get Event History: This function retrieves historical events based on criteria such as context mask, event name, filter expression, date/time range, sort order, and maximum result count. This function is essential for accessing raw event data, including granules generated by the Granulation Engine.

  • Get Variable History: This function loads historical values of a context variable as a table of timestamps and values. It offers options for specifying the date/time range, sort order, maximum result count, and whether to include the raw data table or just its encoded string representation.

  • Summary Function: This powerful function performs on-the-fly aggregation of historical data from multiple sources (events, variables, or statistical channels) over custom time periods and with user-defined aggregation methods. It offers advanced options for handling counter-type values, out-of-range values, and time zone adjustments. The summary function is particularly useful for generating reports and building complex chart datasets.

Data Visualization and Analysis

Historical data retrieved using these functions can be directly used by various Iotellect modules:

  • Charts: Build interactive charts displaying historical trends based on raw or aggregated data. Flexible configuration options allow for customizing axes, legends, tooltips, and series types (gauge, counter, derive, absolute) to effectively visualize the data.

  • Reports: Generate printer-friendly reports incorporating historical data.

  • Queries: Use queries to perform advanced analysis on historical data by filtering, sorting, grouping, and aggregating the retrieved values. The query results can then be used as a data source for other contexts, such as reports or charts.

  • Scripts: Process historical data with user-defined scripts written in any of a number of powerful programming languages, including Java and Python. This allows for custom calculations, transformations, or complex analytical tasks, extending the built-in functionality of Iotellect.

  • Machine Learning: Utilize historical data as training datasets for Machine Learning models. This enables predictive analytics, anomaly detection, and other data-driven applications, enhancing the intelligence and automation capabilities of your SCADA system.

Leveraging the Expression Language for Advanced Data Processing

Central to the power and flexibility of the Iotellect historian is the platform's expression language. The expression language provides a concise and powerful way to perform complex calculations, transformations, and analysis on historical data, whether during data retrieval, visualization, or reporting.

Here's how the expression language enhances working with historical data:

By combining the power and flexibility of the expression language with the comprehensive data acquisition, aggregation, and retrieval capabilities described above, Iotellect provides a powerful and versatile historian solution that surpasses the limitations of classic SCADA historians, letting you to extract maximum value from your data.

  • Custom Data Extraction: When retrieving data with the Get Variable History or Get Event History functions, use expressions to select specific fields, filter rows based on criteria, and perform calculations on the retrieved values. This allows for precise data extraction and avoids loading unnecessary data into memory.

  • On-the-Fly Aggregation: The summary function leverages expressions to define custom aggregation logic. This goes beyond the standard aggregation functions (average, min, max, etc.) offered by the RRD, enabling calculations such as weighted averages, custom statistical metrics, or string concatenations over arbitrary time periods.

  • Data Transformation and Enrichment: Use expression language functions to transform data types, format values, create calculated fields, and enrich data with information from other contexts. This enhances the analytical value of the data and simplifies its integration with other systems.

  • Conditional Logic and Branching: Implement conditional logic and branching within expressions to handle different scenarios, filter out-of-range values, or perform custom calculations based on specific criteria. This makes for flexible and adaptive data processing workflows.

  • Integration with other modules: Expressions bridge the historian functionality with other Iotellect modules such as alerts, reports, and models. For example, use an expression in an alert trigger to raise an alert if a specific trend in historical data is detected.

Was this page helpful?