Arpe Technologies旗下工具FastBCP发布1.6版本,新增backup_mssql连接类型,支持直接从SQL Server备份文件(.bak)中提取表数据并转换为Parquet格式 1。该功能允许用户在不运行SQL Server实例、不执行数据库恢复操作且无需提供凭证的情况下完成数据导出 1。这一特性主要解决了大数据量场景下的效率瓶颈,以及传统方式中跨版本恢复受限和负载隔离困难等问题 1。
在性能方面,针对一个320 GB的生产环境备份文件,首次打开耗时2分29秒,后续访问仅需0.3秒;使用24线程将一个包含1.75亿行的表导出至Parquet格式耗时29.6秒 1。另一项基于TPC-H SF10数据集的笔记本测试显示,在未压缩备份条件下,暖启动导出拥有1500万行的orders表仅需1.5至2秒,速度是实时adbc_mssql源的2.3到2.4倍 1。此外,针对宽列存储表的内存占用也得到显著优化,在8线程配置下,峰值内存从早期的42 GB降低至5.9 GB 1。
该工具通过专用驱动ArpeMSBackup直接解析备份二进制文件来实现上述功能 1。目前,它支持SQL Server 2017、2019、2022及2025版本的完整备份(Full)和仅复制备份(Copy-only),但不支持差异备份和日志备份 1。在压缩与加密兼容性上,FastBCP支持MS_XPRESS和ZSTD压缩格式,但无法处理TDE加密备份及Intel QuickAssist压缩备份 1。配合其FastTransfer组件,导出的Parquet文件可被加载至任意版本的SQL Server或PostgreSQL中,从而突破原生RESTORE命令的版本限制 1。典型应用场景包括单表恢复、数据迁移以及数据湖填充等 1。
Arpe Technologies has released version 1.6 of FastBCP, a tool that introduces the ability to export table data directly from SQL Server backup files (.bak) to Parquet format without requiring database restoration or an active SQL Server instance 1. The new release adds a backup_mssql connection type that utilizes a dedicated driver, ArpeMSBackup, to parse backup binary files directly 1. This approach addresses issues related to cross-version recovery and workload isolation by bypassing the need for server credentials or running SQL Server processes during extraction 1.
The utility supports Full and Copy-only backups from SQL Server versions 2017, 2019, 2022, and 2025, but does not accommodate differential or log backups 1. Regarding compression, FastBCP handles MS_XPRESS and ZSTD formats while excluding TDE-encrypted backups and those using Intel QuickAssist compression 1. Performance metrics from a 320 GB production environment backup indicate an initial open time of 2 minutes and 29 seconds, which drops to just 0.3 seconds for subsequent accesses 1. Furthermore, exporting a table containing 175 million rows to Parquet took 29.6 seconds when utilizing 24 threads 1. In a separate benchmark involving the TPC-H SF10 orders table (15 million rows) on a laptop with an uncompressed backup, warm-start exports completed in 1.5 to 2 seconds, achieving speeds 2.3 to 2.4 times faster than live adbc_mssql sources 1. Optimizations in version 1.6.5 specifically target wide-column storage tables, reducing peak memory usage from 42 GB to 5.9 GB when operating with 8 threads 1.
Beyond extraction, the integrated FastTransfer component allows users to load the resulting Parquet files into any version of SQL Server or PostgreSQL 1. This feature effectively circumvents the version limitations inherent in native RESTORE operations 1. Key application scenarios for this technology include single-table recovery, general data migration, and populating data lakes 1.
评论
还没有评论,欢迎留下第一条。