LakeSail offers a fully managed, Rust‑native data and AI platform that runs Apache Spark workloads without the JVM overhead.
Funding
Funding not disclosed
Founders
Product
Problem
Data and AI teams using Apache Spark face high compute costs and performance bottlenecks due to the JVM runtime, including startup latency, garbage‑collection pauses, and costly serialization between Spark and Python workloads. Migrating to a faster engine typically requires code rewrites, data migration, or complex infrastructure changes.
Solution
LakeSail provides a fully managed, Rust‑native data and AI platform that implements the Apache Spark Connect protocol, allowing existing PySpark, Spark SQL, Delta Lake, and Iceberg code to run unchanged. By replacing the JVM with a native Rust engine built on Apache Arrow and DataFusion, the service delivers up to 8× faster query execution and up to 94% lower compute cost while scaling to zero between jobs. The platform runs inside the customer’s AWS account, preserving data locality and open formats, and adds native Python UDF support, AI‑focused workloads, and an agent layer without additional integration effort. Users switch engines by updating a single configuration line, gaining performance and cost benefits without code modifications or data migration.
Target Audience
LakeSail targets data engineering, analytics, and AI teams that run Spark workloads on AWS and need higher performance and lower cost without rewriting existing pipelines.
Features
- Spark Connect compatibility: drop‑in endpoint replacement runs all existing Spark DataFrame, SQL, Delta Lake, and Iceberg operations unchanged
- Rust‑native Sail engine built on Apache Arrow and DataFusion for zero‑GC, instant startup, and SIMD‑accelerated vectorized execution
- Native in‑process Python UDFs and AI workloads, eliminating JVM‑Python serialization overhead
- Full support for open lakehouse formats (Iceberg, Delta Lake) with no data movement; operates in the customer’s AWS VPC
- Stateless, auto‑scaling workers with scale‑to‑zero, providing transparent compute‑hour billing and no minimum spend
- Integrated agent layer and lakehouse branching for LLM and other AI agents to interact with data safely and reversibly