How to fill a Sift Board

Quick start

Send rows in with a webhook

Use this when the tool already knows something happened and can post it: a CI job that finishes, a deployment script, a cron job, a monitoring alert. Nothing is polled and there is no schedule to tune.

Add a webhook view. The board gives you a web address ending in a view id, and a token that starts with sb_. Post JSON to that address with the token in a header, and the rows you post land in that view.

• The address looks like https://...webtrigger.atlassian.app/...?view=abc123
• Send the token in the X-Sift-Token header (or as Authorization: Bearer). A token in the address itself is refused, because addresses end up in logs
• Send an array of objects, or {columns, rows}. The keys become the columns, in the order they first appear
• A value can be plain, or {value, time, note, color}. The time and note are what make stale values fade later
• In replace mode each post is the whole table. In update mode each post updates matching rows by a key column and leaves the rest alone
• The reply says how many rows were received and stored, and truncated: true if rows beyond this install's limit were dropped
• Only the token the board currently shows is accepted. Click New token and the old one stops working at once
• A refused post (wrong view, retired token, a body that is not JSON or not rows) is recorded on the view, so a page editor sees what to fix in Edit board

Keep the token out of your build logs. It is the only thing standing between the internet and that view, so treat it like any other CI secret. Only people who can edit the page can see it, in the editing panel.

CSV import and paste

For data that is not coming from a tool at all: a spreadsheet someone maintains, an export from a system with no API, a one-off list you want to filter and colour rather than reformat by hand.

• Import a CSV file, or paste rows straight into the view
• The first row is treated as the header, so those names become your columns
• Paste again to replace the rows. The columns, buttons and colour rules stay as they were
• Saving the view without touching the paste box keeps its rows. Clear the box and save to empty the view

A table from a Jira query

Add a Jira view, type the JQL you already use, and pick the fields you want as columns. The query runs on a schedule and the results fill the view.

• Any JQL that works in Jira works here, for example project = OPS AND status != Done ORDER BY priority
• It reads Jira through Atlassian’s own API, so nothing leaves your Atlassian account
• It can only see the issues the person who set up the view can see
• Narrow the query in JQL rather than pulling everything and filtering afterwards

This is also the answer to the one thing Sift cannot do: a Jira Issues macro table is not in the page body, so Sift cannot see it. A Sift Board fetches the issues itself and draws its own table.

Scheduled connectors

A connector is an outgoing call on a timer. The board asks the service for data, on the interval you set, and keeps the view current between asks. Five minutes is a normal interval.

There are ready-made setups for GitHub Actions, Jenkins, GitLab, Azure DevOps, CircleCI, Bitbucket, SonarQube, PagerDuty, Datadog, Grafana, ServiceNow, TestRail, Linear and Google Sheets. Each one asks for the few fields it needs, usually a project or repository name and a token, and works out the columns for you.

• Anything else with a JSON API works through the plain JSON connector: a URL, a header and the path to the list of rows
• The URL must be a public https address. Private, internal and loopback addresses are refused when you save, since Confluence could not reach them anyway
• Pick which fields become columns, and which of them get filter buttons
• If a fetch fails, the last good rows stay on the board and fade instead of disappearing, and the reason is shown to page editors in Edit board
• A fetch gives up after 20 seconds, so a host that hangs cannot hold up the other views
• Anything with no API at all can still push in through a webhook or the Sift Agent

Be aware of what this means for data flow: a connector is the one part of Sift Board that talks to the outside world. The privacy policy says exactly what goes where.

The Sift Agent

For a database, a file, or the output of a command. Confluence cannot reach your database, and you should not want it to. The Agent is a small program you run inside your own network. It runs the query and posts the rows out to a webhook view. Nothing connects in from outside, and your database password never leaves your environment.

• Postgres, MySQL, SQL Server, SQLite, MongoDB, CSV and JSON files, or any command that prints rows
• One settings file, then npx sift-agent run to keep it going on a schedule
• Read credentials from environment variables, for example connection: ${PG_URL}, so nothing secret is in the file
• Run it wherever the data already is: a jump box, a build agent, a laptop for a quick test

Views, filters and colours

One board holds as many views as you want, and each view is a button at the top. Build health from Jenkins, open incidents from Postgres and this sprint from Jira can be three buttons on one page.

• Every value in a column becomes a filter button with a count, the same as in Sift
• Colour rules work the same way too: Result: fail=red, pass=green
• Words like pass, fail and warning are coloured on their own, with no rule needed
• Sort by clicking a header, click a cell to filter by it, add a totals row
• Export CSV of what is on screen, or copy it into Excel
• A Refresh button pulls the latest rows without reloading the page
• Only people who can edit the page can change the board. That is checked by Confluence on every change, not just used to show or hide the controls
• A board belongs to the page it was made on. Copying the page gives the copy an empty board of its own, never the original’s rows or tokens

What happens to your tokens

A connector needs a credential for the service it calls. Those are the most sensitive thing a board holds, so they are handled the way you would want them handled.

• Stored encrypted in Atlassian’s Forge storage, inside your own site
• Write-only: after you save a token it is never displayed again, not to you and not to me
• A token only ever goes where it was entered for. Change the URL, tool, repository or auth of a view and its saved token is cleared; enter it again with that change
• Replace or delete one at any time. Deleting a view deletes its token, and uninstalling the app removes them all
• Give each connector a token with the least access it needs. A read-only token is almost always enough
• Webhook tokens are different: they let a tool write to a view, so they are shown to page editors in the editing panel, and can be rotated there at any time

Get help

Email aravind@arumugam.co.uk. You get a reply within two working days. Bugs, ideas, and connectors you wish existed are all welcome. If anything on this page does not match what you see in the app, tell me and I will fix the page.

Also useful: the live board, Sift for Confluence, privacy and terms.