dmux/minilake 项目维护者发布了名为 MiniLake 的开源工具,这是一个用于在本地模拟 Databricks API 环境的模拟器 1。该工具集成了 SQL、Delta Lake 和 Spark 的真实执行能力,允许开发人员在不依赖云端账户或产生费用的情况下,通过标准 SDK、CLI、Terraform 或 LLM Agent 进行本地开发和测试 1。MiniLake 采用 MIT 许可证免费开源 1。
在技术实现上,MiniLake 使用 DuckDB 处理 SQL 和元数据,并通过 Docker 容器运行真实的 spark-submit 作业 1。其 Unity Catalog 功能提供完整的目录、架构、表和资源卷 CRUD 操作,每个目录对应一个独立的 DuckDB 数据库文件 1。此外,该工具内置 Model Context Protocol (MCP) 服务器,包含 67 个工具和 8 个资源,专为 AI Agent 优化 1。兼容性方面,MiniLake 支持 databricks-sdk、Databricks CLI、Terraform、Asset Bundles 及 LLM agents 1。用户可通过 pip install minilake 或 docker run -p 8000:8000 ghcr.io/dmux/minilake:latest 命令进行安装 1。
需要注意的是,MiniLake 主要针对单个本地开发者设计,并非多租户共享服务 1。其权限 API 仅为模拟,不反映真实工作区的授权逻辑 1。
MiniLake is an open-source tool released as a local emulator for the Databricks API environment. 1 It enables developers to execute real SQL, Delta Lake, and Spark jobs locally without relying on cloud accounts or incurring costs. 1 The project supports compatibility with standard development tools, including databricks-sdk, Databricks CLI, Terraform, Asset Bundles, and LLM agents. 1
Technically, MiniLake uses DuckDB to handle SQL queries and metadata management. 1 Real spark-submit jobs are executed within Docker containers to ensure authentic processing behavior. 1 Installation options include running pip install minilake or using the Docker command docker run -p 8000:8000 ghcr.io/dmux/minilake:latest. 1 For Unity Catalog functionality, the tool provides full CRUD operations for catalogs, schemas, tables, and resource volumes. 1 Each catalog corresponds to an independent DuckDB database file. 1 Additionally, MiniLake includes a built-in Model Context Protocol (MCP) server optimized for AI agents, featuring 67 tools and 8 resources. 1
The software is free and distributed under the MIT license. 1 It is designed specifically for single local developers and is not intended for multi-tenant shared services. 1 Furthermore, its permission APIs are simulations that do not reflect the authorization logic of real workspaces. 1
评论
还没有评论,欢迎留下第一条。