Verteego Doc
  • Getting started
    • About Verteego
    • Sample Use Cases
    • Concepts
  • Data
    • Introduction
    • Datasources
      • URL connector specification
    • Datasets
  • Pipelines
    • Forecasting Pipelines
      • Getting started
      • Configuration
        • Identifying and preparing data
          • calculated_cols
          • cols_type
          • date_col
          • normalize
          • preprocessing
          • prediction_resolution
        • Configuring the Forecasting Algorithm
          • algo_name
          • algo_type
          • algorithm_parameters
          • fit_parameters
          • conditional_algorithm_parameters
        • Building the Training and Prediction Set
          • column_to_predict
          • features
          • input_prediction_columns
        • Using Hyperparameter Tuning for the Model
          • tuning_search_params
          • hyperparameter_tuning_parameters
        • Evaluating the Results of the Forecast
          • scores
        • Modifying the results of the forecast
          • postprocessing
      • Calculators
        • External source
          • get_from_dataset
          • weather
        • Mathematic
          • aggregate_val_group_by_key
          • binary_operation
          • count_rows_by_keys
          • hierarchical_aggregate
          • mathematical_expression
          • unary_operation
          • Moving Average (EWM)
        • Machine Learning
          • pca
          • clustering
          • glmm_encoder
          • one_hot_encode
          • words_similarity
        • Transformation
          • fillna
          • fill_series
          • case_na
          • interval_index
          • constant
          • cyclic
          • replace
        • Temporal
          • bank_holidays_countdown
          • bankholidays
          • date_attributes
          • date_weight
          • day_count
          • duration
          • events_countdown
          • seasonality
          • tsfresh
    • Optimization Pipelines
      • Getting started
      • Configuration
      • Constraints
        • Unary Constraint
        • Binary Constraint
        • Aggregation Constraint
        • Order Constraint
        • Multiplicative Equality Constraint
        • Generate constraints from a Dataset
  • Apps
    • About Apps
    • Recipes
      • Pipelines
      • Datasets
  • Users
    • User roles
  • Best practices
    • Performance analysis and ML model improvement
  • Developers
    • API
    • Change logs
Powered by GitBook
On this page
  • Understanding Datasources
  • Data Import and Export
  • Seamless Data Flow
  • Historical Data Tracking
  1. Data

Introduction

Data Integration and Management in Verteego

PreviousConceptsNextDatasources

Last updated 1 year ago

Verteego is fundamentally designed around the concept of , which serve as the core repositories for your data. These datasets are not static; they are dynamic extracts sourced from various data environments such as your production databases, integration databases, or data lakes.

Understanding Datasources

are essential components in Verteego. They act as the foundational storage solutions, often referred to as connectors, that facilitate the flow of data into and out of the platform. Once a datasource is defined within Verteego, it becomes the gateway through which datasets are extracted and utilized for your AI-driven applications.

Data Import and Export

The process begins with the Import (Extraction) of datasets from established datasources. If appropriate permissions are set on your datasources, Verteego can also directly export the processed data back to your operational environments. This capability ensures that data remains fluid and actionable, enhancing the adaptability and responsiveness of your business operations.

Seamless Data Flow

This integration architecture ensures that Verteego can seamlessly interact with your existing data infrastructure, supporting a wide array of storage solutions and maintaining the integrity and security of your data throughout its lifecycle in the platform. Whether it's pulling raw data for initial analysis or pushing refined insights back into production, Verteego handles each step with precision and efficiency.

Historical Data Tracking

In Verteego, each refresh of a dataset from its datasource is meticulously recorded, preserving a comprehensive history of all dataset versions. This feature not only enables robust data audits and supports data governance but also enhances the explainability of decisions. Each decision recommendation generated by Verteego is linked to the specific dataset version used, making it clear why certain recommendations were made at a particular time. This traceability ensures that decisions are transparent and accountable, fostering trust and confidence in the AI-driven insights provided by Verteego.

Datasets
Datasources