STILTS

Starlink Tables Infrastructure Library Tool Set

Latest (see Version History)

Version 3.5 released 7 August 2024

New: TAP service authentication: Commands tapquery, tapskymatch, taplint and tapresume now allow login to TAP services with optional authentication (auth=true)
I/O handler enhancements, bugfixes, build/source modernisation

Version 3.4-10 released 29 February 2024

New: Support for HAPI time series services: Data from Heliophysics Data API services can be read using the HAPI input handler and table scheme.

What is STILTS?
Commands
Documentation
Mailing Lists
Downloads
Further Information

What is STILTS?

The STIL Tool Set is a set of command-line tools based on STIL, the Starlink Tables Infrastructure Library. It deals with the processing of tabular data; the package has been designed for, but is not restricted to, astronomical tables such as source catalogues. Some of the tools are generic and can work with multiple formats (including FITS, VOTable, CDF, ECSV, CSV, PDS4, Parquet, MRT, Feather, GBIN, SQL and ASCII), and others are specific to the VOTable format. STILTS is the command-line counterpart of the GUI table analysis tool TOPCAT. The package is robust, fully documented, and designed for efficiency, especially with very large datasets.

Facilities offered include:

format conversion
crossmatching
plotting
column calculation and rearrangement
row selections
data and metadata manipulation and display
sorting
statistical calculations
histogram calculation and aggregate functions
data and service validation
access to remote data services including Virtual Observatory

A powerful and extensible expression language is used for specifying data calculations. These facilities can be put together in very flexible and efficient ways.

For tasks in which the data can be streamed, the size of table STILTS can process is effectively unlimited. For other tasks, tables of a few million rows and a few hundred columns usually do not present a problem.

STILTS is written in pure Java (Java SE 8 or later; versions prior to STILTS 3.2 were Java SE 6), and can be run from the command line or from Jython, or embedded into java applications. The STILTS application is released under the GNU GPL, though much of the library code is licensed under the LGPL or other less restrictive licenses - see LICENSE.txt.

Commands

STILTS currently consists of generic table processing commands:

tcopy - Table format converter
tpipe - Generic table pipeline processing utility
tmatch2, tskymatch2 - Two-table crossmatchers
tmatch1 - Intra-table crossmatcher
tmatchn - Multi-table crossmatcher
tjoin - Trivial side-by-side multiple-table joiner
arrayjoin - Adds table-per-row data as array-valued columns
tgridmap - N-dimensional histogram calculator with table output
tcube - N-dimensional histogram calculator with FITS array output
tgroup - Grouped aggregate function calculator
tcat, tcatn - Multiple-table concatenaters
tmulti, tmultin - Multiple-table container writers
tloop - Test table creation utility

plotting commands (as well as three deprecated ones):

plot2plane - Plots on 2d Cartesian axes
plot2sky - Plots on celestial axes
plot2cube - Plots on 3d Cartesian axes
plot2sphere - Plots in spherical polar space
plot2corner - Plots a Corner Plot, a grid of scatter plots and histograms
plot2time - Plots 2-d axes with Time horizontally

VOTable-specific commands:

votcopy - VOTable encoding translator
votlint - VOTable validity checker

Virtual Observatory/external data service access commands:

cone - Cone-search like queries (including SIA and SSA)
tapquery, tapresume, tapskymatch - TAP service clients
cdsskymatch - match local table against VizieR/SIMBAD using CDS X-Match service
taplint - TAP service test suite
datalinklint - DataLink validator
regquery - Registry Query
coneskymatch (formerly multicone, now somewhat deprecated) - Match local table with one behind a Cone Search/SIA/SSA service

sky pixel-related commands:

tskymap - Generate HEALPix sky density maps
pixfoot - Generate Multi-Order Coverage (MOC) maps
pixsample - Sample from a HEALPix pixel data file

SQL-specific commands:

sqlskymatch (formerly sqlcone) - Match local table with one in an SQL database
sqlclient - JDBC-based SQL command-line client
sqlupdate - Updates data in existing cells of an RDBMS table

and some miscellaneous items:

server - HTTP server which executes STILTS commands
calc - Quick expression evaluator
funcs - Documentation browser for expression language functions
xsdvalidate - Validates against XML schema

See also the Commands by Category section of the manual.

More commands and facilities may be added in the future.

Documentation

The STILTS documentatation is provided in the user document, SUN/256. This is available in the following formats:

Mailing Lists

Two mailing lists exist for STILTS, TOPCAT and related software:

topcat-user@jiscmail.ac.uk: Public list for questions, answers, discussions, bugs, comments, ...
topcat-announce@bristol.ac.uk: Low-volume read-only list for release announcements and news items.

See the mailing list page for more information.

If you have queries or support enquiries you can send them to topcat-user. However, I'm still happy to answer mail directly to me if you prefer that.

Downloads

You can download STILTS from one of the following links:

Main jar file stilts.jar (16.1M)
Invocation script stilts
Standalone JyStilts jar file jystilts.jar (57.7M) to run JyStilts without an independent Jython installation
Documentation zip file (6.4M)
Full source code repository at http://github.com/Starlink/starjava/ (see ttools subdirectory)
Source code zip file (3.6M)
Zip archive of individual jar files stilts_jars.zip (14.8M) (may be useful for those using STILTS classes as a library)
LICENSE.txt file discussing licensing
Archive of previous, and perhaps pre-release versions

The stilts.jar file contains all the classes required to run STILTS. If you run "java -jar stilts.jar" you will see a help message; on Unix you also have the choice of downloading the stilts script into the same directory and using that for convenience. For more details on invoking the tools, read SUN/256.

It is also possible to run STILTS commands against an installation of the full Starjava set or of TOPCAT (run "topcat -stilts ... or "java -jar topcat-full.jar -stilts ...") since the relevant classes are present in each case.

If you have MacOS and homebrew, then installing TOPCAT with "brew install --cask topcat --no-quarantine" will install the stilts script along with the TOPCAT application.

The current release of STILTS is v3.5, released 7 August 2024. The full Version History is available in the user document.

Further Information

Products related to STILTS are:

STIL: Public API underlying the functions of STILTS
TOPCAT: Graphical user tool for interactive table manipulation

If you have any comments, questions, requests, bugs etc, please either post to the topcat-user list or contact me direct:

Author: Mark Taylor, Astrophysics Group, School of Physics, University of Bristol
Email: m.b.taylor@bristol.ac.uk
STILTS WWW page: http://www.starlink.ac.uk/stilts/

If you use this software in published work, then citing paper 2006ASPC..351..666T would be appreciated.

And finally...

Q: What do you get if you combine all the Starlink Java table handling applications?
A: TOPCAT STILTS.
Thanks to Proporta and Malcolm Currie for that.