Introducing RouterKV: TypeScript key-value wrapper for libSQL.

We’re very happy to announce Router KV, a new open-source Typescript key-value store built on libSQL (Turso’s fork of SQLite).

What is a KV?

A Key-Value Store is a simple database consisting of key-value pairs, where each key is unique and maps to an individual value. In contrast, a Relational Database usually features multiple tables with multiple rows and columns linked together by foreign keys.

KV stores are ideal for unstructured data models, performance and scalability, whereas Relational Databases are suited for applications with complex relationships between data models and allow for complex querying such as joining and aggregation.

Why We Built Router KV#

While working on Router, an app we are currently developing that removes the limitations found within no-code/low-code platforms, we found ourselves needing a KV store that we could deploy on a per-tenant basis with minimal technical overhead. Additionally, we wanted global replication and the option to host embedded replicas on users' virtual machines.

We evaluated other KV solutions, such as Cloudflare Workers KV and Deno KV, but these didn’t meet our requirements for various reasons.

Sub-hosting#

One of our goals with Router is to provide users with their own isolated environment, so it was important to also offer users their own KV instance. Whilst Deno KV is available with sub-hosting, we opted to run Deno on VMs rather than on the edge. When self-hosting Deno, you do have the option to use DenoKV; however, this is backed by SQLite running in-process, useful for testing & development, but this did not meet our goals when it came to scalability.

Embedded replicas#

Since Router runs on VM(s), we also wanted to offer support for embedded replicas so that read operations require one less hop. This reduction in time taken for read operations is very important, especially for server-side rendering. Looking at Cloudflare and Deno's offerings, while both are server-less and, therefore, allow read operations to be directed to the nearest instance, neither option currently allows for embedded read replicas.

DenoKV does support self-hosting a read replica via an S3 bucket, you can learn more here.

Addressing our target audience#

Another issue we found with the other solutions is that certain aspects of the APIs were overly technical. Router is designed to be friendly for less technical folk, and for this reason, we were doubtful about how understandable the implementation was for features such as the pagination controls on the list function or the chaining of methods to implement transactions.

Since our target audience is no-code and low-code developers, we wanted to build a KV that was accessible to all. For this reason maintaining our own KV library allows us to ensure it is user-friendly and fit for purpose.

Turso to the Rescue#

Turso is an Open Source, edge-based, distributed database powered by libSQL. LibSQL, which was developed by the Turso team, is a fork of SQLite which also allows contributions, an advantage over SQLite since this allows for tailored optimisations and enhancements to suit the specific needs of users and applications. Allowing contributions also ensures that libSQL remains dynamic and adaptable to the evolving landscape of edge computing.

Read-Replicas#

Turso allows for read-replicas, which are replicas of the full KV store on different servers in different regions. This allows for read operations to be made to the nearest server, thereby reducing latency for all read operations. Any write operations are still directed to the primary node, and Turso handles all synchronisation of changes between the primary node and the replicas. By utilising read replicas we can horizontally scale, decreasing the individual read load of each database, allowing for higher total read throughput.

Embedded Read-Replicas#

LibSQL features the ability to create local embedded read replicas, this is huge because it allows read operations to be handled locally. Similarly to how hosted read replicas speed up read times, embedded read replicas reduce the latency for read operations even more. As for write operations, libSQL handles these by writing to the Turso database, and libSQL also handles the syncing between the two, with the option to define a sync interval or allow it to happen programmatically.

Examples of Router KV in Action#

Installation (Deno & Node)#

RouterKV officially supports Deno and Node please refer to the documentation for installation instructions.

Basic Operations#

The below examples use twitter style user handles to illustrate unique values. In a production app you should use unique & static user identifiers.

Set#

Sets a key-value pair in the KV table.

await kv.set("scores:@gillmanseb", 42);

Output

{ k: "scores:@gillmanseb", v: 42, created_at: "...", updated_at: "..." }

Get#

Retrieves the value for a specified key if it exists.

await kv.get("scores:@gillmanseb");

Output

{ k: "scores:@gillmanseb", v: 42, created_at: "...", updated_at: "..." }

List#

Lists entries in the KV table based on prefix and other options.

await kv.list("scores:");

Output

{
  data: [
    { k: "scores:@gillmanseb", v: 42, created_at: "...", updated_at: "..." },
    { k: "scores:@willneeteson", v: 41, created_at: "...", updated_at: "..." }
  ],
  meta: {
    limit: 100,
    offset: 0,
    reverse: false,
    orderBy: "k",
    total: 2
  }
}

Delete#

Deletes a specific key-value pair if it exists.

kv.delete("scores:@gillmanseb");

Output

void

Delete All#

Deletes all key-value pairs with keys beginning with prefix in the KV table.

kv.deleteAll("scores:");

Output

void

Advanced Operations#

Transaction#

Executes a series of operations within a transaction, ensuring all or nothing execution.

async function claimBonus(
  user: string,
) {
  try {
    const bonusValue = 100;
    
    const score = await kv.transaction<number>(async (tx) => {
      // check if the user has claimed the bonus already
      const hasClaimedBonus = await tx.get(`bonuses:${user}`) as KvQueryResult<number>;
      if (hasClaimedBonus) {
        throw new Error("Bonus already claimed!");
      }

      // get the users current score
      const usersScore = await tx.get(`scores:${user}`) as KvQueryResult<number>;

      // calculate the users updated score
      const currentScore = usersScore?.v ?? 0
      const updatedScore = currentScore + bonusValue;

      // update the users score
      await tx.set(`scores:${user}`, updatedScore);

      // mark the user as having claimed the bonus
      await tx.set(`bonuses:${user}`, true);

      // return the updated score
      return updatedScore;
    });

    console.log(
      "Bonus applied!",
      `Your new score is ${score}`,
    );
  } catch (error) {
    console.error(error.message);
  }
}

Conclusion#

In summary, we're releasing a new KV library utilising libSQL, which allows for simple and fast KV operations, ideal for our Low-Code audience. Whilst we made it with our own application, Router, in mind, we think that many people could benefit from it. So please check out our SDK, and if you have any issues, please let us know.

And that brings our first overview of Router KV to an end, we're so excited to see what you will do with it! Before you leave, consider giving the repository a ⭐️ or joining the mailing list, we've got some very exciting announcements coming.

Appendix#

• RouterKV Repository: https://github.com/RouterDev/router-kv

• Router: https://router.dev/

• Turso: https://turso.tech/

• LibSQL docs: https://docs.turso.tech/libsql