Today, I am happy to announce the release of Swarm64 DA 4.0, database acceleration extensions for Postgres. Here are the highlights:

• We’ve tripled the acceleration effect on Postgres, mainly by adding additional parallelism and support for SQL JOINs acceleration.
• FPGA support is now optional; you can accelerate Postgres with OR without using an FPGA.
• Postgres text filtering is now accelerated.
• We patched pg_dump to backup and restore foreign table (FDW) data. The patch was contributed, and committed in PostgreSQL 13.

You can try Swarm64 DA for free, and we hear that it is a great way to spend your time in quarantine 🙂

• Launch a Swarm64 DA-accelerated Postgres instance from the AWS Marketplace
• Request a trial copy of Swarm64 DA if you want to run it somewhere else.

No FPGA required (but optional)

If you know anything about Swarm64, then you probably know that we’ve always been “that FPGA acceleration for Postgres company.” We’re still that, but starting with 4.0, you don’t need an FPGA in your server to benefit from Swarm64 database acceleration.

Why did we make FPGA support optional starting in version 4.0?

We made FPGA optional because a number of Postgres users asked whether we could accelerate query performance without requiring them to modify their server hardware. Meeting that request was totally doable!

Swarm64 DA accelerates Postgres quite a lot even without the FPGA. Swarm64 DA extends Postgres with greater parallelism, columnar indexing, data compression (which both reduce I/O), and other C and C++ level performance engineering we’ve done. All of that can happen with no FPGA (CPU-only), and increases query performance by at least an order of magnitude for all use cases we tested in our labs.

When should CPU- vs. FPGA-acceleration be used?

As I mentioned, CPU-acceleration will likely boost your query performance by at least 10x. If one or more FPGAs are present on the server, we load them with a software image that increases parallelism and compresses data more aggressively. The feature set is the same for Swarm64 DA’s CPU and FPGA acceleration with one exception…text filtering acceleration is only supported by FPGA today.

Therefore, as a rough guideline, we recommend users consider using FPGAs if they need extra acceleration to handle:

• Larger data volumes (10s of TBs)
• Larger concurrent user/client requirements
• Larger mixed workloads on single servers (i.e., CPUs handling transaction processing, and FPGAs serving reporting/query requests)
• Acceleration of text filtering and analytics in your SQL

New: SQL JOIN acceleration

We enhanced our query planning, hash join execution, and memory utilization to enable SQL JOIN acceleration. Swarm64 DA will accelerate inner-, left-, semi-, and anti-joins. If you’re trying to join large tables with multi-millions of rows each, Swarm64 DA can likely help. We saw PostgreSQL TPC-H benchmark times improve in our labs by about 50%. We’ll blog about our latest TPC-H benchmarks in detail in an upcoming post.

New: text filtering acceleration

Swarm64 DA 4.0 query planner and execution supports the SQL “LIKE” text search command. We had users doing log analytics, and accelerating Postgres text search saved them from having to switch from Postgres to a search database. It’s great for security and other search-intensive and text analytics use cases.

As I mentioned above, the text filtering is only supported if an FPGA is present on the server. Pattern matching of the “LIKE” command is pushed down to the FPGA hardware and works as part of the processing flow, executing this complex operation without any additional cost in latency and CPU cycles. As a result, text can be processed at around 20 GBs per second of raw text on each FPGA, leading to dramatic filtered text query performance gains of 10x or more.

Foreign table data backup/restore in pg_dump

Since Swarm64 DA stores data in foreign tables (partitioned, compressed, columnarized), we wanted a way to make database administration easier for our customers. So, in Swarm64 DA 4.0, we developed a patched version of pg_dump that allows you to dump the content of a foreign table using the --include-foreign-data option. The patch was contributed and committed to PostgreSQL 13 (yay!).

What’s next?

What’s next for us? We’re happy to have nearly tripled Swarm’s acceleration of Postgres in this release, and we know there’s still room for improvement. So…we are continuing to focus on making Postgres even faster.

For those hardware acceleration fans out there, we are working hard on new hardware-level optimizations including support for Samsung SmartSSD computational storage, Intel Optane persistent RAM (huge transaction per second speed up with that), and support for cloud FPGA instances as they arise.

What’s next for you? Hopefully staying well, getting some fresh air, and finding time to give Swarm64 DA a try!