Central Pipeline
Aggregate and process data at any scale within a unified platform.
Pay-As-You-Go
$0.028
per ingested GB*
- Streamlined Data Transformation
Convert unstructured logs into structured formats using built-in scripts for easier analysis. - Comprehensive Analytics
Extract essential attributes to enable precise filtering, aggregation, and in-depth reporting. - Proactive Data Desensitization
Mask sensitive identifiers—like names or IDs—before storage, preserving privacy by default.
*Billed daily, the cost is based on the amount of data processed by the central pipeline.
Multi-Year/Volume Discounts Available
Common Questions
- What is a pipeline, and what types of pipelines are available?
A pipeline is a lightweight scripting language running on DataKit, designed for custom parsing and modification of collected data. By defining parsing rules, pipelines can finely slice various types of data and convert them into structured formats to meet specific data management needs. Truewatch supports two types of pipelines:
- Local Pipeline: Executes during data collection, requiring DataKit collector version 1.5.0 or higher.
- Central Pipeline: Runs after data is uploaded to the platform center.
- What is the difference between central and local pipelines?
- Local pipelines process data during collection, allowing for real-time data parsing and modification before forwarding. In contrast, central pipelines handle data after it has been uploaded to the platform, enabling more complete data processing and analysis at the central level.
- How can I write and debug a pipeline script?
- Truewatch provides a built-in debugging tool within DataKit to assist in writing pipeline scripts. Users can specify the name of the pipeline script and input sample text to verify extraction success, facilitating efficient script development and testing.
- Can pipelines handle all data types collected by DataKit?
- Yes, starting from DataKit v1.4.0, pipelines can process all data types collected by DataKit, including Log, Metric, Trace, Network, and Object data. This allows for consistent and customizable data processing across various data categories.
- Does Truewatch support AI-generated pipelines for automated data processing?
- Yes, Truewatch provides AI-powered assistance to help generate pipelines automatically. By analyzing your data patterns and processing requirements, the AI can suggest or create pipeline scripts tailored to your specific needs.