Import

Data › Import lets people add or update records from a CSV file or pasted text. The table owns the whole screen: reading the file, matching its columns to the table's, converting values, the review and the progress. Your application only stores rows, through its existing create and update actions or a server-side bulk write. React and Vue behave the same.

Import appears in the Data menu, between Export and Connect, when the table can create rows (allowCreate with actions.create), update them (allowEdit with actions.update) or import in bulk (actions.import.importRows). Set table.import: false to hide it; see Configuration.

What the user sees

Choose a file

The first step offers a drop zone and Choose a file for a .csv, .tsv or .txt file, and a text box to paste CSV text, for example cells copied from a spreadsheet. Sources declared by your application follow, such as a CRM or another table; see Other sources.

• Encoding: the file is read as UTF-8, or as Windows-1252 when it is not valid UTF-8 (older Excel exports). A byte order mark is ignored.

• Separator: comma, semicolon, tab or vertical bar, detected from the first lines. The Separator select shows the detected one and can force another.

• Headers: First row is headers is on by default. Turn it off when the file starts with data; its columns are then named "Column 1", "Column 2"…

• Quoted values, escaped quotes, line breaks inside quotes and both CRLF and LF line endings follow RFC 4180. Blank lines are ignored.

Match the columns

Each column of the file gets a row with a sample value ("e.g. Alpha launch") and a select listing the table's columns plus Ignore. The table proposes a match for every column:

1. By name: the file's header against each column's header and id, with accents, case, spaces, - and _ ignored. When two columns share a name, the one whose type fits the values wins.

2. By values: a header no name matches goes to the unused column that converts the most of its first 20 values, at least 80% of them. Text columns never win on values alone, and a tie leaves the file column ignored.

3. Otherwise the column is ignored.

Each column can be changed by hand. A table column takes one file column: choosing a column already taken moves it and ignores the other file column. A badge next to each row shows the column's type, or how many of its first 20 values won't convert ("3 won't convert").

Match existing records by picks the key column. File rows whose key matches a record update that record; the others are added. It defaults to Don't match (create all), unless a mapped column is an id (id, key or Yayaw ID).

The Preview shows the first five rows as they will be written. An invalid cell is highlighted, with its error on hover.

Review and import

Review looks up the keys, then shows how many rows will be added, updated or have errors ("2 to add", "1 to update", "2 rows with errors") with the first three errors, such as "Row 5, Status: not one of the options". Rows are counted like the file's lines, the header being row 1.

Skip rows with errors is on by default: the other rows are imported. Turning it off blocks Import until the file is fixed. Back returns to the mapping and Cancel closes the screen without writing anything.

Import writes the rows in batches with a progress bar ("Importing… 50/120"). Stop ends the import between two batches. A row that fails does not stop the others; only a sign-in or permission error stops the import. The result lists the rows added, updated and failed, with the first failures, then Done or Import another file. The table refreshes as soon as a row was written.

On phones, the screen opens in the Data drawer and each select opens as a full-screen list.

How values convert

Each cell is converted for the type of the column it goes to:

• An empty cell is empty (null). A required column rejects it on rows that create a record. On rows that update a record, empty cells leave the existing value unchanged.

• An unknown select option is an error, unless actions.import.allowNewOptions is true: the value is then kept as a new option.

• A key value repeated in the file is an error on its later rows, so a file never updates the same record twice.

• Selection, actions and custom columns are never offered.

The column mapping screen

The mapping step is a reusable component, ColumnMapping (components/toolbar/column-mapping.tsx in React, components/toolbar/ColumnMapping.vue in Vue). The send screen of Connect destinations renders through it too, in the other direction: table columns to the target's fields. Both screens match names with the same rules, from the shared field-matching.ts.

Import actions

Without any configuration, a CSV import checks keys against the table's records and writes each row through create or update. Declare import on your table actions (getTableActions in React, get-table-actions in Vue) to change that. Every field is optional:

import: {
  csv: true, // offer CSV files and pasted text (default true)
  sources: [], // other sources, after CSV
  importRows: (batch) => importProjects(batch), // server-side bulk write
  lookup: ({ columnId, keys }) => findProjectIds({ columnId, keys }), // key matching
  allowNewOptions: false, // keep unknown select options
  batchSize: 50, // rows per write (default 50)
},

• importRows(batch, context) receives up to batchSize rows: { creates: { rowIndex, values }[], updates: { rowIndex, values, id }[] }. values holds the converted values keyed by column id, and id the record to update. It returns { created?, updated?, failures?: { rowIndex, message? }[] }. When provided, it writes every row instead of create and update.

• A thrown error fails every row of its batch with its message, and the import goes on. An error carrying status 401 or 403, or code unauthorized, forbidden or invalid_credentials, stops the import with "You are not allowed to import into this table."

• lookup({ columnId, keys }) returns the record ids by key value, Record<string, string>, for the key column chosen on screen. keys are the file's key values, converted like the column. Without it, the table loads every record through list with an empty query (100 per page), or uses the rows it holds when there is no list, and reads each record's id with the table's getRowId, else id.

• context is the view's context, as for custom destinations: viewId, query, columns, selectedRowIds, url and loadRows.

The types (TableImportActions, ImportSource, ImportBatch, ImportBatchResult) are exported by import-flow.ts and import-model.ts: components/ui/yayaw-table/utils/ in React, components/ui/yayaw-table-vue/ in Vue. They have no framework dependency, so your server code can import them too.

Import on your server

With importRows, rows are written in bulk on your server instead of one request per row. Check permissions and validate the values again there: the browser has already converted them, but a request can be forged. This Next.js server action writes a batch of projects and reports failures per row:

"use server";
import type {
  ImportBatch,
  ImportBatchResult,
  ImportRowFailure,
} from "@/components/ui/yayaw-table/utils/import-model";
import {
  canImportProjects,
  findProjectIdsByColumn,
  insertProjects,
  requireUser,
  updateProjects,
  validateProject,
} from "@/server/app";

const KEY_COLUMNS = new Set(["id", "name"]);

export async function importProjects(
  batch: ImportBatch
): Promise<ImportBatchResult | { error: "forbidden" }> {
  const user = await requireUser();
  if (!(await canImportProjects(user))) {
    return { error: "forbidden" };
  }
  const failures: ImportRowFailure[] = [];
  const valid = <T extends { rowIndex: number; values: Record<string, unknown> }>(
    rows: T[],
    mode: "create" | "update"
  ) =>
    rows.filter((row) => {
      const problem = validateProject(row.values, mode); // e.g. "Price must be positive"
      if (problem) {
        failures.push({ rowIndex: row.rowIndex, message: problem });
      }
      return !problem;
    });

  const creates = valid(batch.creates, "create");
  const updates = valid(batch.updates, "update");
  // One statement per batch; `updateProjects` only touches the user's records
  // and returns the ids it could not update.
  await insertProjects(user, creates.map((row) => row.values));
  const missing = new Set(
    await updateProjects(user, updates.map(({ id, values }) => ({ id, values })))
  );
  for (const row of updates) {
    if (missing.has(row.id)) {
      failures.push({ rowIndex: row.rowIndex, message: "Record not found" });
    }
  }
  return {
    created: creates.length,
    updated: updates.length - missing.size,
    failures,
  };
}

export async function findProjectIds(request: { columnId: string; keys: string[] }) {
  const user = await requireUser();
  if (!KEY_COLUMNS.has(request.columnId)) {
    return {};
  }
  // { "Alpha launch": "prj_12", … } for the records the user can see.
  return await findProjectIdsByColumn(user, request.columnId, request.keys);
}

The functions imported from @/server/app stand for your own code. Next.js does not pass an error's code from a server action to the browser, so the action returns { error } and the table actions turn it into an error the import recognizes:

import type { TableImportActions } from "@/components/ui/yayaw-table/utils/import-flow";
import { findProjectIds, importProjects } from "./import-projects";

export const projectImport: TableImportActions = {
  importRows: async (batch) => {
    const result = await importProjects(batch);
    if ("error" in result) {
      // `forbidden` stops the import with the permission message.
      throw Object.assign(new Error("Import not allowed"), { code: result.error });
    }
    return result;
  },
  lookup: findProjectIds,
  batchSize: 200,
};

Add import: projectImport to your table actions next to list, create and update. In Vue, import the types from @/components/ui/yayaw-table-vue/import-flow and @/components/ui/yayaw-table-vue/import-model, expose the same functions as Nuxt server routes and call them with $fetch.

Other sources

sources adds choices after CSV, such as a CRM, a Notion database or a Google Sheets tab. Each is { id, label, description?, load(context) }, and load returns a table of strings or CSV text:

sources: [
  {
    id: "crm",
    label: "CRM contacts",
    description: "Contacts updated this week",
    load: async () => {
      const contacts = await loadCrmContacts(); // your server function
      return {
        name: "CRM contacts",
        headers: ["Name", "Email", "Company"],
        rows: contacts.map((contact) => [contact.name, contact.email, contact.company]),
      };
    },
  },
],

A source returning { name?, text } is read like a pasted CSV. The rows then go through the same mapping, review and import. Set csv: false to offer only your sources. A source that throws shows "The source could not be read.", or the permission message for a sign-in or permission error.

Connect destinations whose connector can pull are listed too, as "From Google Sheets" ("Imports its records and keeps them linked."). They open the connector screen with the direction set to import instead of this screen: the records are mapped field by field, previewed and imported through the connector's sync, and stay linked to the target for later syncs. Back and Done return to Import. table.sync: false hides these sources.

Labels

Every label of the screen has built-in English and French text and can be overridden with import.<key> translations, one key at a time: import.title renames the Data menu entry and the screen, import.keyColumn the key select, import.skipErrors the checkbox, and so on. The translations page lists every key.

Set table.import: false in the configuration to hide Import while keeping create and update for the table's own forms.