Concepts Overview¶
Learn the mental model behind Seeknal and understand how its components work together.
What Are Concepts?¶
Concepts are the fundamental ideas and patterns that underlie Seeknal's design. Understanding these concepts will help you use Seeknal effectively and troubleshoot issues.
Core Concepts¶
Pipeline Builder Workflow¶
The fundamental workflow for creating and deploying data pipelines:
YAML vs Python¶
Seeknal supports two approaches for defining pipelines:
- YAML: Declarative, configuration-driven
- Python: Programmatic, flexible
Virtual Environments¶
Isolated workspaces for safe development and testing:
- Plan changes before applying
- Test in isolation
- Promote to production safely
Change Categorization¶
Understanding the impact of changes:
- BREAKING: Requires manual intervention
- NON_BREAKING: Safe to apply automatically
- METADATA: Documentation-only changes
Parallel Execution¶
Run independent tasks concurrently for faster pipelines:
- Automatic parallelization
- Dependency management
- Performance optimization
Advanced Concepts¶
Point-in-Time Joins¶
Prevent data leakage in ML features by joining data as it appeared at prediction time.
Second-Order Aggregations¶
Create hierarchical aggregations and multi-level rollups.
When to Read Concepts¶
- Before starting: Get oriented with key ideas
- During learning: Deepen your understanding
- Troubleshooting: Understand what's happening under the hood
- Best practices: Learn recommended patterns
Learning Path¶
New to Seeknal? Start with these concepts in order:
- Pipeline Builder Workflow - Understand the core workflow
- YAML vs Python - Choose your approach
- Virtual Environments - Safe development practices
Experienced user? Jump to advanced topics:
- Point-in-Time Joins - ML feature engineering
- Second-Order Aggregations - Advanced analytics
- Parallel Execution - Performance optimization
Related Topics¶
- Glossary - Definitions of key terms
- Building Blocks - Detailed component reference
- Getting Started - Hands-on tutorials
Next: Explore Pipeline Builder Workflow or return to Choosing Your Path