Insert Data

Insert the data in a file into a table.

To be able to insert into a table, it must have been created with the /create endpoint.

• The data in the files must conform to the schema, and must use the same column names as the schema.
• One successful /insert request consumes 10 credits.
• The maximum request size is 1.2GB

Consistency

A request to this endpoint has two possible outcomes: Either all of the data in the request was inserted, or none of it was. It’s not possible for parts of the request data to be inserted, while the rest is not inserted. In other words, please use and trust the status codes that the endpoint returns. A status code of 200 means that the data in the request was successfully inserted. If you get any other status code, you can safely retry your request after addressing the issue that the error message indicated.

Concurrent requests

A limited number of concurrent insertion requests per table is supported. However, there will be a slight performance penalty as we serialize the writes behind the scenes to ensure data integrity. Larger number of concurrent requests per table may result in an increased number of failures. Therefore, we recommend managing your requests within a 5-10 threshold to maintain optimal performance.

Supported filetypes

CSV files (Content-Type: text/csv)

CSV files must use a comma as delimiter, and the column names in the header must match the column names of the table.

JSON files (Content-Type: application/x-ndjson)

These are files where each line is a complete JSON object which creates one table row. Each line must have keys that match the column names of the table.

Data types

DuneSQL supports a variety of types which are not natively supported in many data exchange formats. Here we provide guidance on how to work with such types.

Varbinary values

When uploading varbinary data using JSON or CSV formats, you need to convert the binary data into a textual representation. Reason being, JSON or CSV don’t natively support binary values. There are many ways to transform binary data to a textual representation. We support hexadecimal and base64 encodings.

base64

Base64 is a binary-to-text encoding scheme that transforms binary data into a sequence of characters. All characters are taken from a set of 64 characters.

Example: {"varbinary_column":"SGVsbG8gd29ybGQK"}

hexadecimal

In the hexadecimal representation input data should contain an even number of characters in the range [0-9a-fA-F] always prefixed with 0x.

Example: {"varbinary_column":"0x92b7d1031988c7af"}

Authorizations

x-dune-api-key

string

headerrequired

The API key of your team or user.

Path Parameters

namespace

string

required

The namespace of the table to insert into (e.g. my_user).

table_name

string

required

The name of the table to insert into (e.g. interest_rates).

Body

The body is of type file.

Response

200 - application/json

rows_written

number

The number of rows that were written to the table.

bytes_written

number

The number of bytes that were written to the table.