Next Previous Up Contents
Up: Release Notes
Releases to date have been as follows:
Version 0.1b (29 April 2005)
- First public release
Version 0.2b (30 June 2005)
- Added Times func class for MJD-ISO8601 time conversions.
- Fixed bug when doing NULL_ test expressions on first column in table.
Version 1.0b (30 September 2005)
- This is the first non-experimental release of STILTS, and
it incorporates major changes and backward incompatibilities
since version 0.2b.
- The parameter system has undergone a complete rewrite;
there is now only a single command "
invoked using the
stilts script or the
stilts.jar jar file, and the various tasks are
named as subsequent arguments on the command line.
Command arguments are supplied after that.
The new invocation syntax is described in detail elsewhere in
this document. As well as invocation features such
as improved on-line help, optional prompting,
parameter defaulting, and more uniform access to common features,
this will make it more straightforward to wrap these tasks
for use in non-command-line environments, such as behind a
SOAP or CORBA interface, or in a CEA-like execution environment.
- A new command
tmatch2 has been introduced.
This provides flexible and efficient crossmatching between
two input tables. Future releases will provide commands for
intra-table and multi-table matching.
- A new command
tcat has been introduced, which
allows two tables to be glued together top-to-bottom.
This is currently working but very rudimentary - improvements
will be forthcoming in future releases.
- A new utility command
calc has been introduced,
which performs one-line expression evaluations from the
- The following new filter commands for use in
and other commands have been introduced:
addskycoords: calculates new
celestial coordinate pair from existing ones
(FK4, FK5, ecliptic, galactic, supergalactic)
replacecol: replaces column data,
using existing metadata
badval: replaces given 'magic'
value with null
replaceval: replaces given 'magic'
value with any specified value
tablename: edits table name
stream parameter of
allows you to write filter commands in an external file, to
facilitate more manageable command lines.
Wildarding for column specification is now allowed for some
- New functions for converting time values between different
coordinate systems (Modified Julian Date, ISO-8601,
Julian Epoch and Besselian Epoch).
- New RANDOM special function.
- SUN/256 has undergone many changes. Much of the tool
documentation is now automatically generated from the code
itself, which goes a long way to ensuring that the documentation
is correct with respect to the current state of the code.
Version 1.0-1b (7 October 2005)
- Fixed jar file manifest bug which prevented working on Java 1.5
Version 1.1 (10 May 2006)
- A number of new features and capabilities have been introduced:
- The new tcube command
calculates N-dimensional histograms (density maps) from N columns
of an input table and writes the result to a FITS file.
- The following new filters
have been added:
stats filter provides the same information as
stats output mode, but allows much more
flexible use of the results. It can also calculate many new
quantities, including quantiles, skew and kurtosis.
meta filter provides the same information as
meta output mode, but allows much more
flexible use of the results.
assert filter provides in-pipeline logical
uniq filter collapses multiple adjacent identical
or similar rows.
sorthead filter provides a (usually) more
efficient method of doing what you could previously do
colmeta filter adds/modifies metadata for selected
check filter checks table in stream - for debugging
Additionally usage of the
sort filter has been changed
so that it can now do everything that
sortexpr used to
be able to do;
sortexpr is now withdrawn.
- The following new output modes
have been introduced:
and usage of the following has been modified:
plastic mode broadcasts the table to
one or all registered PLASTIC listeners.
cgi mode writes the table to standard output in a
form suitable for output from a CGI script.
discard mode throws away the table.
topcat mode now attempts to use PLASTIC
(amongst other methods) to contact TOPCAT.
meta modes are mildly
deprecated in favour of the corresponding new filters
Other new features
- New IPAC table format input handler added.
csv-noheader format variant output handler
functions introduced for more control over visual appearance
of numeric values.
- Experimental facilities for automatically generating a CEA
application description file.
Bug fixes and minor improvements
- Now copes with 'K'-format FITS binary table columns
- Improved, though still imperfect, retention of table-wide
metadata in VOTables.
- Distinctions between null and false values in boolean columns are
handled more carefully for FITS and VOTable files.
- Efficiency improvement when writing FITS-plus format
(now only requires a maximum of two passes rather than
three of the input rows).
- Added the
system property which can
optionally work around a bug in some input streams
("Resetting to invalid mark" errors).
- Fixed a bug in Cartesian matching which failed to match
if the required error in any dimension was zero.
- Fixed erroneous reports about unknown
utype attributes of TABLE element in
- When joining tables, column name comparison to determine
whether deduplication is required is now case-insensitive.
- Error message improved when no automatic format detection
is attempted for streamed tables.
istream=true is now less likely to cause a
"Can't re-read stream" error.
Version 1.2 (7 July 2006)
- New features for permitting column-oriented storage
colfits format, new
sideways") have been introduced.
These can provide considerable efficiency improvements for
certain tasks when working with very large (and especially wide)
New VO commands
- Added two new commands for querying Virtual Observatory services:
These tasks are experimental and may be modified or renamed in
- Makes multiple cone search queries to the same service
- Queries the VO registry
transpose filter added.
- Added flux conversion functions (Jansky<->magnitude).
- ISO-8601 strings now permit times of 24:00:00 as they should.
Version 1.2-1 (3 August 2006)
- Tab-Separated Table (TST) format now supported for reading and
- Added ICRS coordinate system for
- TUCDnn header cards now used in FITS files to transmit UCDs
- Efficiency improvements for column-oriented access.
Version 1.3 (5 October 2006)
- The old
tcat command has been replaced by more capable
Between them these provide concatenation of an unlimited number of
homogeneous or heterogeneous input tables.
Additional columns may be added to indicate which of the input tables
given output rows originated from.
Parameter value indirection
- Certain parameters
cmd and friends)
may now be specified in the form "@filename".
This indicates that the value for the parameter is to be obtained
by reading it from the named file.
This is useful if a very long value is required for the parameter
in question. The
script parameter of
tpipe has therefore been
withdrawn, since it did just the same thing.
- Direct access to the MySpace virtual file system is now provided
by use of
- Time conversion functions between MJD and Decimal Year have
fromHex numeric conversion
functions have been added
- The HTML version of SUN/256 now uses CSS to provide better
- The Output Modes and
Processing Filter sections are
now split into subsections to make the table of contents
- The Command Reference section now
has only one level of subsection listed in the table of
contents to make it clearer.
Other new features and improvements
-J flag to
for passing flags directly to Java.
- Added new
out parameter to
-ifshape flags to
exact match mode in
now copes with array-valued columns.
force parameter to
as a workaround for some broken services.
- Added Sample (as opposed to Population) Standard Deviation/Variance
calculation options to the
- Improved CEA description file output - now contains details of
all tasks rather than just a few, as well as various
improvements in documentation etc.
- Fixed erroneous complaints from
utype attribute on RESOURCE elements.
- Fixed a couple of minor crossmatching bugs (which wouldn't
have affected results).
Version 1.3-1 (Starlink Hokulei release)
- New command
- Output to MySpace can now be streamed, if running under J2SE1.5 or
- Slight changes to parameters for
- Fixed bug in handling of single quotes in FITS file metadata.
-bench flag to
- Various scalability improvements for use with very large (Tb?)
- Improved efficiency for
output formats (now one-pass not two-pass).
- Improved CEA app-description file, including especially option lists
for things like input and output formats.
- Added README.cea file to distribution.
- Fixed problem which could mis-report VOTable out of memory errors
as Broken Pipe.
- Added Vega<->AB magnitude conversion constants to
duptag parameters to tmatch2
task for customised renaming of columns with duplicated names.
- Added hyperbolic trig functions to
- Added cosmology distance calculations in class
funcs task, a browser for
expression language function documentation.
-checkversion to list of
Version 1.3-2 (6 July 2007)
- Added optional
table parameter to
(for access to table parameters).
- Can use table parameter names in expressions using
param$ notation (Section 10.2).
- Can reference columns/parameters by UCD by using
in expressions (Section 10.1)
and as column identifiers (Section 6.2).
- Improved deduplication of column names when joining tables.
- Fix error in output of FITS table
cards - write them as numeric not string values.
- Improve error message for broken CSV files.
- Modified JDBC handling so that MySQL and PostgreSQL do not run out
of heap memory when streaming large datasets for input.
Think I've done the same for SQL Server, but this is not tested.
- Improve error reporting in the presence of a deficient JVM
(such as GNU
- Add locale-specific
formatDecimalLocal functions in
functions in class
- Fix bug which was causing NullPointerExceptions
Version 1.3-3 (4 Sep 2007)
- Experimental, and currently undocumented,
task introduced, along with some classes in package
uk.ac.starlink.ttools.cone designed for library use by
AstroGrid DSA code.
- CEA description of
parameter now has
minoccurs=0, since that can be true
for exact matches.
Version 1.3-4 (10 Sep 2007)
Version 1.3-5 (30 Oct 2007)
-stderr flags to
- Some bugs fixed in generation of CEA
- Documentation provided for
- Fixed error in
Version 1.4 (6 December 2007)
- This version provides more cross matching functionality.
Added to the existing
command are new tasks:
stripped down version of
tmatch2 for ease of use when
matching with sky coordinates.
internal matcher, finds groups of objects within a table.
finds group or multiple-pair matches between multiple (>2)
Two tasks have been renamed for improved clarity and consistency:
There has also been some enhancement and rationalisation of
parameters for all table join tools (
tmatch* as well as
- All table join commands now use similar
suffix* parameters to control renaming of duplicated
columns in output tables (note this replaces the old
duptag* parameters in
- Crossmatching tasks have a new
which allows you to configure whether progress is reported to the
copycols parameter of
sqlskymatch now defaults to "
(include all columns from input table in the output).
Section 7 of the manual has been somewhat
rearranged and improved.
- FITS reader now imports table HDU header cards as table parameters.
- CeaWriter can now output CEA service definition XML config file as
well as app-description file (experimental - may be withdrawn).
- Embedded spaces in output ASCII format table column names
are now substituted with underscores.
- Fix a bug which caused an infinite number of dots to be printed
when attempting a crossmatch with an empty input table.
votlint handling of TABLEDATA-type
These are now split up into strings by counting characters rather
than using whitespace delimiters.
I think it's doing the right thing now.
Version 1.4-1 (28 January 2008)
New RDBMS-related features
- New command
which is a general JDBC-based SQL command-line client.
- New command
which allows updates to existing rows in SQL tables.
- Some changes to
- choice of options for how to write to the database output table,
controlled by new associated parameter
- associated parameter
- associated parameter
db for consistency with other commands
Local and service-based matching command enhancements
- New parameter
scorecol added to
which controls adding a new column to match output tables
containing a goodness-of-match value.
- New parameter
parallel added to
which allows multiple cone searches to be carried out in
- New parameter
erract added to
coneskymatch which controls response to
isolated failures in individual cone search queries.
- Improved error reporting (reasons for errors are now reported
even without the
- Add new help option
help='*' which prints help for
all parameters of a task at once.
- Added (mostly undocumented)
for reducing verbosity level.
- Minor improvements to CEA app-description.
- Downgraded from WARNING to INFO log messages about
the (extremely common) VOTable syntax error of omitting
a FIELD/PARAM element's
Version 1.4-2 (26 March 2008)
progress parameter to
emptyok parameter to
- Fixed pair matching performance bug (slower if tables were not
given in the right order) introduced at v1.4.
- Fixed null handling error in
- Fixed error in
stats filter cardinality value
- Fixed minor bugs in suffix addition for matching commands
- Removed unformatted XML output in
- Try to avoid exponential format in cone search URLs
(some endpoints seem to require fixed point format).
- Minor CEA fixes.
Version 2.0b (23 October 2008)
- This version contains two new major items, plotting and server mode.
Both work, but are missing desirable features and have not had extensive
testing in the field, so should be considered experimental at this stage.
- Two table plotting commands are now provided:
See also the new Plotting
section in the manual.
- A new command
provided which allows STILTS commands to be executed via HTTP.
One purpose of this is to facilitate server-side use of the
plotting commands co-located with data to generate on-the-fly
graphical summaries of server-held datasets.
Smaller enhancements and bugfixes
- Efficiency improvements (~25%? in both CPU time and memory usage)
for HEALPix-based sky crossmatching
(thanks to Nikolay Kouropatkine at Fermilab for a new version of
the PixTools library).
- New class Arrays
added to algebraic functions.
- New Appendix Commands by Category
added to manual.
(max/min ignoring blank values) in class
- Sexagesimal field identification for ASCII input files
is less stringent
(now permits minutes or seconds equal to 60).
- Minor CEA fixes.
- HEALPix bug fix
(PixTools bug fix update).
- Fix bug in use of
Version 2.0-1 (23 December 2008)
- Can reference columns/parameters by Utype by using
in expressions (Section 10.1)
and as column identifiers (Section 6.2).
- Non-alphanumeric column names may now be used for algebraic expressions
in the special case that the expression is just the value of a
regquery command has changed in implementation,
data access, and output format. It now queries VOResource1.0
registries rather than the very out of date registry protocol which
was used in earlier versions.
Version 2.0-2 (9 January 2009)
- Added new samp output mode which passes
the generated table to clients using the SAMP protocol.
- Updated the topcat output mode to use
SAMP as one way of communicating with a running TOPCAT.
-version flag now reports starjava subversion revision
as well as other items.
Version 2.0-3 (27 March 2009)
- Fits BINTABLE TZERO/TSCAL value reading improvements:
- Columns with integer TZERO values now read as integers
rather than floating point values where possible.
This includes unsigned longs ('K'), which were previously
represented as doubles with lost precision.
Unsigned longs which are too large however (>263)
are read as nulls.
- Byte-valued columns can now be written out by
handler as signed byte values (TFORM=B,TZERO=-128)
rather than signed shorts (TFORM=I).
- More comprehensive testing.
- Fixed bug in calculating value scaled double ('D') values.
- Fixed bug in typing value for scaled float ('E') arrays.
- Fixed bug which caused registry queries (
to fail for Java 1.6.
- Fix minor bugs in detail of
votlint's validation tests
(VOTABLE element content model,
INFO and PARAM and FIELD required attributes).
- Report application name and version in User-Agent header of outgoing
- The fixed length Substring Array Convention for string arrays
is now understood for FITS binary tables.
- Minor SAMP bugs fixed (JSAMP upgraded to 0.3-1).
Version 2.0-4 (17 July 2009)
- Work around J2SE mark/reset bug when loading table direct from URL.
- Produce null rather than nonsense results from sky coordinate
conversions with unphysical latitudes
- Produce null rather than questionable results from sexagesimal
conversions with mins/secs out of range.
- Fix two bugs in
votcopy: XML processing instructions
garbled on output, and pathnames in
inappropriately flattened in hrefs attribute values.
Version 2.0-5 (2 Oct 2009)
- VOTable 1.2 supported.
votlint can now validate VOTable documents following
the (provisional, 2009-09-29 PR) VOTable 1.2 standard.
- Namespacing of VOTable documents made more intelligent, and
configurable using the
votlint now checks that the correct XML namespaces are
- Be more careful in XML, including VOTable, output;
fix VOTable output encoding to be UTF-8,
and ensure no illegal XML characters are written.
- HTML table output is now HTML 4.01 by default
(includes THEAD and TBODY tags).
parse* string->numeric conversion
functions now cope with leading or trailing whitespace.
- Work around illegally truncated type declarations in IPAC tables.
- Fix a bug which caused the first table in a multi-table file
(FITS or VOTable) to be used in streaming mode, even if
a subsequent one was requested.
- Bug fixed in crossmatching output: entries which should have been
null were sometimes written as non-null (typically large negative
numbers) in FITS and in non-TABLEDATA VOTable output.
This affected cells in otherwise non-nullable columns
where the entire row was absent. The previous behaviour is not
likely to have been mistaken for genuine results.
Version 2.1 (6 November 2009)
Note that the STIL API used by this release has changed in some
backwardly incompatible ways, and may change further.
If you're using STILTS as a library rather than an application
you might want to wait for a later release when the API has settled down.
coneskymatch can now match using SIA and SSA
services as alternatives to Cone Search ones
(see its new
- Fixed an obscure bug which could under rare circumstances cause
truncation of strings with leading/trailing whitespace read
from text-format files.
- A new
startable.storage policy "
is now the default. This should mean running out of memory less
often. The old behaviour can be restored by giving the new
-memory command line flag.
Version 2.1-1 (21 December 2009)
- Plotting commands can now output to PDF as well as existing graphics
- New filter
- Fixed internationalisation bug which could cause
coneskymatch to fail in locales that use ","
for a decimal point.
- Significant performance improvements related to the case of VOTable
documents containing many tables.
Version 2.1-2 (24 March 2010)
This is a jython (i.e. Python, though not CPython)
interface to the STILTS commands.
It is believed to be fully working, but somewhat experimental -
feedback is encouraged.
- Considerable performance and scalability improvements to the
For several common regimes, using default settings,
memory use has been decreased by a
factor of about 5, and CPU time reduced by a factor of about 3.
- Add optional tuning parameters to crossmatch commands
Experimentation with these can lead to significant performance
improvements for given matches.
- Fixed a crossmatch bug; it was giving a possibility of
match assignments when pair matching in crowded fields.
Crossmatch results thus may differ between earlier versions
and this one. Both are reasonable, but the newer behaviour is
more correct. In non-crowded fields, there should be no change.
- Further performance improvement for VOTable documents with
very many TABLEs.
- Memory management adjusted further - default (Adaptive) storage policy
now uses direct allocation (=
for intermediate-sized buffers
to avoid running out of java heap space.
- New option "
This allows you to get an output table with exactly one row for
each row of the input table.
- New flag
-memgui to monitor memory usage during runs.
- Add new filter
- Add new functions to
array functions for constructing arrays,
and new aggregating functions
- Syntax of the crossmatching commands'
has changed; it now has an additional option which will write
limited profiling information as well as logging as the match
ylabel parameter to
sequential filters have
respectively. This provides a better idea of what they do.
Since they are only useful for debugging, it is unlikely that this
will break anyone's existing code.
- New filter
random introduced which converts tables
to random-access if necessary.
- Document previously undocumented
legend parameter to
- Matching commands
matcher parameters can now accept
MatchEngine implementation classes as
- Classes are now distributed as a zip of jars
stilts_jars.zip) as an alternative to
the monolithic jar file (
This may be more appropriate for those using STILTS classes in a
framework that contains other third party class libraries.
- Adjusted the way that data types are read from JDBC databases.
Date, Time and Timestamp typed columns will now be converted to Strings
which means they can be written to most output formats
(previously they were omitted from output tables).
- STILTS no longer attempts to communicate with TOPCAT using SOAP.
TOPCAT's SOAP interface has been deprecated since v2.1 (2006),
so this isn't likely to cause trouble, and it permits removal of
SOAP (Axis) classes from the application jar file, saving several
megabytes and reducing potential version clash problems.
- Fix bug in code for handling very large mapped FITS files.
This was causing fatal read errors in some cases.
Version 2.2 (6 August 2010)
- New capabilities for multi-table I/O have been introduced:
There are some additional enhancements:
- New multi-table output tasks
These currently just copy multiple input tables to a single
multi-table container file (e.g. Multi-Extension FITS or
multi-TABLE VOTable). Future releases may generalise the output of
multi parameter introduced for
tmulti tasks to pick up all tables in a
multi-table container file.
- New JyStilts functions
for multi-table I/O.
And a number of bug fixes:
- Added experimental name-resolution filter
this currently uses Sesame.
- Added filter
repeat, which repeats table rows a
given number of times.
- Recognise unofficial column type "
long" in IPAC format
- Better behaviour (warn + failover) when attempting to read large
files on 32-bit OS or JVM.
- Efficiency warning now issued for large compressed FITS files.
- Upgraded PixTools HEALPix library to 2010/02/09 version.
This fixes a bug that could theoretically cause deficient crossmatch
results, though I haven't managed to produce such errors.
- Fixed bug in TST table output.
- Fixed bug in FITS-plus metadata output
(table parameters were getting lost).
- Corrected literature references in Fluxes conversion class
documentation (thanks to Mattia Vaccari).
- Fixed bug in CSV file parsing that could ignore header row in
absence of non-numeric columns.
- Shape and ElSize metadata items now correctly reported by
- Fixed JyStilts bug when supplying an empty string for a parameter
Finally, from this release STILTS requires version 1.5 (a.k.a. 5.0)
of the Java J2SE Runtime Environment; it will no longer run on
version 1.4, which is now very old. I don't expect this to cause
compatibility issues for anyone, but I'm interested to hear if that's
not the case.
Version 2.2-1 (23 December 2010)
- Storage management improvements; removed restriction on large
(>2Gb) non-FITS datasets in some circumstances.
- Efficiency improvement in sequential mapped access to large FITS
- Fix so FITS tables >2Gb can provide random access in 32-bit mode
(though slower than 64-bit).
- FITS files now store table names in EXTNAME (and possibly EXTVAR)
- Window placement for the few GUI tasks should now behave a bit more
like platform norms, rather than sitting in the top left hand
- HTML table output now writes cell contents which look like URLs
in HTML <A> tags.
- Basic authorization (
table URLs handled.
- Fixed file pointer int overflow bug in FITS MultiMappedFiles.
Version 2.3 (9 May 2011)
- The new commands
tapresume have been introduced.
These provide support for the Table Access Protocol (TAP),
and allow freeform queries in an SQL-like language to be
made to remote databases.
- Random Groups HDUs are now tolerated, though not interpreted,
within FITS files.
soapout parameter to
stdev functions to
- Upgrade to JSAMP v1.2.
- Improve text rendering in
funcs window display.
- Attempt case-sensitive matching before case-insensitive for
replaceval filter to work with Infinities.
Bug fixes and workarounds
- JDBC table input handler now effectively downcasts
BigInteger/BigDecimal types to Long/Double.
The PostgreSQL JDBC driver seems to use the Big* types routinely
for numeric values (which I don't think it used to do).
- Add workaround for J2SE bug
which could cause errors when reading compressed FITS files.
- Fix FITS character handling bug which could cause corrupted FITS
files on output in presence of non-ASCII characters.
- Fix (some) JDBC connection leaks.
- Add missing parameters
Version 2.3-1 (30 June 2011)
- Added new command
This is a validator for TAP (Table Access Protocol) services.
It is only likely to be useful to people developing or operating
- ASCII table parsers now understand python-friendly
- Added new constants to expression language
- Fixed a significant bug in sky crossmatching.
If all points in a table were on one side of the RA=0 line,
but the error radius extended across that line, matches on the
other side could be missed. Matches could also be missed if
different tables used different conventional ranges for RA
(e.g. -180..180 in one case and 0..360 in another).
This fix may in some, but not most, cases result in slower matching
coneskymatch cone search verbosity parameter so that
VERB=3 is not erroneously ignored.
Version 2.4 (27 October 2011)
- Two new asymmetric match options
best2 have been added
find parameter in the pair matching commands
They correspond to finding the best match in table B for each row
in table A, and in crowded fields often provide more intuitive
semantics than the previous symmetric
(in non-crowded fields there is generally no difference).
This replicates the matching performed by some other tools,
- New matchers have been added to permit matching of general
elliptical, rather than just circular, regions in both planar
and sky coordinates; see
- Another new matcher is available for dealing with per-object
errors in Cartesian coordinates (previously per-object errors
could only be handled in sky coords); see
- Semantics of the
matcher have changed slightly.
Expression language functions:
- Algebraic functions involving angles are now mostly available
using degrees as well as radians.
Coords class has been replaced by
classes providing sky coordinate functions,
and a new class
provides normal degree-based trigonometric functions
alongside the radian-based versions in
Some of the old function names have changed to make clear that they
use radians and not degrees.
This change should be much more convenient in most cases;
sorry it's taken so long to get round to.
- Add new
join function is added to the
to combine all the elements of an array into a string.
- There are several bugfixes and changes related to the
TAP validator tool
taplint, mostly thanks to bug
reports etc from the TAP community:
- Improve test logic for record limiting queries.
- Errors no longer reported (e.g. E-Qxx-CNAM)
for unexpected TAP_SCHEMA table column ordering
(when running query stage but no metadata acquisition stages).
- Add new stage MDQ, which checks query result columns for all
tables against declared metadata.
- Add check of versioned and unversioned LANG variants.
- Now uses corrected upload ID
as per most recent TAPRegExt draft.
Bug fixes and minor enhancements:
- Add parameter
allowing pre-send syntax checking of submitted ADQL.
- Add experimental system properties
- Improve resilience of
coneskymatch in the presence of
unreliable or inconsistent DAL services.
- A PARAMref element with no referent in a VOTable
no longer causes an uncaught NullPointerException.
Version 2.5 (28 March 2013)
New coverage-related functionality:
- Add new command
which can sample pixel data from HEALPix table files
(useful for things like Schlegel dust extinction).
which does the same job.
- Add new command
which can generate MOC (Multi-Order Coverage) maps.
- Add MOC-based coverage filter to
when using some Cone Search services (mostly VizieR).
This uses the Multi-Order Coverage map service operated by CDS.
It can make VizieR multi-cone queries much faster by not doing
cone searches that are outside the coverage region of the
catalogue in question.
- Add new class
to the expression language containing MOC-related functions
Other new capabilities:
- Add IPAC table output format.
- Add new class
to the expression language,
containing a method for calculating K-corrections
following the method of Chilingarian and Zolotukhin.
- You can now reference tables in multi-extension FITS files by name
(EXTNAME or EXTNAME-EXTVER) as an alternative to by HDU index.
- VOTable input, output and validation are now supported
for version 1.3 of the VOTable standard.
- The version of the VOTable format used for VOTable output
can now be selected,
by using the system property
Output version is VOTable 1.2 by default.
votlint has been changed so that it handles
different VOTable versions more capably.
Versions 1.1+ are now validated against a schema
(which is how those versions are defined) rather than against
a DTD hacked to do the same job as the schema.
VOTable 1.3 validation is now provided.
votcopy command has a new
parameter to control output version,
and a new
nomagic parameter to control whether
null attributes are removed
- Infinite floating point values are now correctly encoded
in VOTable output ("
-Infinity" as in
votlint is now stricter about floating point
TD element contents.
- VOTable output no longer writes the
attribute by default.
- Add new function
Maths class in expression language.
- Add new
split functions for string splitting
Strings class in expression language.
-utype flags for
utype option for
- Some changes to the
it now works on non-numeric values,
gives the right answer for
and character values,
and returns a blank value rather than the string "null" or "NaN"
for blank inputs.
- Sexagesimal to numeric angle conversion functions now permit the
seconds part of the sexagesimal string to be missing.
- Changes to the IPAC format definition are accommodated:
the "long"/"l" type, which is apparently now official,
no longer generates a warning, and headers may now use
minus signs instead of whitespace.
OBS stage (ObsTAP validation)
- Add more checks to
CAP stage of
Declared languages (including features) and output formats are
- Tidy up error reporting a bit (fewer duplicate nested messages
- PNG graphics output no longer has transparent background.
- Issue a warning for high values of
- Upgrade JSAMP library to version 1.3-3.
- Upgrade Grégory Mantelet's ADQL library to version 1.1.
- Fix serious and long-standing bug
(bad TZERO header, causes subsequent reads to fail)
for FITS output of boolean array columns.
- Fix small but genuine sky matching bug.
The effect was that near the poles matches near the specified
threshold could be missed. The bug was in the PixTools library,
fixed at the 2012-07-28 release.
- Fix bug in
tmatchn group mode which could result in
output rows with columns from only a single table,
i.e. not representing an inter-table match,
- Fix bug which failed when attempting to read FITS files with
complex array columns (
- Fix failure when caching very large sequential tables.
- Fix bug in
filters which could cause truncation of strings in FITS
and possibly VOTable output when the new value was longer
than the previously declared maximum length.
tcatn so that in most cases
output column metadata is compatible with all input tables, not
just the first one in terms of nullability, array shape etc.
- Adjust SQL writer to avoid a type error for MySQL.
- Fix bug in HMS sexagesimal formatting: minus sign was omitted from
negative angles. Now the output is forced positive.
- Cope with 1-column CSV files.
- Use the correct form "
rather than "
for TAP capability unit values.
- Fix error bar rendering bug which could result in diagonal lines
being offset near the edge of plots.
Version 2.5-1 (1 July 2013)
- Add read-only support for
(NASA Common Data Format) files.
- Add Median Absolute Deviation calculation
- Improved handling of HTTP basic authorization.
401s now generate a useful message about the
star.basicauth.* system properties if they
have not been set up.
Bug fixes and minor enhancements
- Fix CSV regression bug introduced at v2.5
- CSV files now work again with MSDOS-style line breaks.
- Fixed FITS output bug which could result in badly-formed
string-valued header cards (no closing quote).
- Source code is now managed by git and not subversion.
The format of the "Starjava revision" string reported by the
-version flag has changed accordingly.
- Output mode
meta now copes better with array-valued
- Implemented fixes to reduce the chance of users inadvertently
overloading external Cone/SIA/SSA services with multicone-like
First, fix it so that abandoned queries are properly terminated,
rather than continuing to hit the server until completion or JVM
Second, implement a sensible default maximum value for the
parallel parameter of
(though this may be adjusted with a system property).
- Quoting behaviour has changed when generating SQL to write to
RDBMS tables. This ought to reduce problems related to mixed-case
identifiers. However, it is possible that it could lead to
unforseen new anomalies.
toString overloads - now works for byte and
boolean values too.
Version 2.5-2 (7 March 2014)
- Add some more colour maps.
- Fix some broken and misdocumented non-table-output JyStilts
- Fix bug which prevented access to long integer array elements
from expression language.
- The Exact matcher now considers scalar numeric values equal if
they have the same numeric value; they are no longer required
to have the same type.
- Fixed a registry client bug which means that the
regquery command can now successfully talk to
the NVO/VAO/STSci registry.
That has been broken since mid-2010.
- Add new command
tloop for generating single-column
tables from a numeric loop variable.
taplint now checks for the right ObsCore ID,
though still recognises the wrong one (got from TAPRegExt),
and warns if found.
- Fix TST input handler so TST files with fewer than 3 columns
can be read.
Nd_cuboid matcher option to match commands.
Version 2.5-3 (4 July 2014)
New and improved functionality:
- Add new command
In most cases (for querying tables that can be found in VizieR)
this can and should be used instead of
coneskymatch - it's much faster.
pixfoot will now guess RA/Dec columns
if relevant parameters are left blank.
- Added new graphics output format
to generate PNG files with transparent backgrounds.
- Upgraded Gregory Mantelet's ADQL library to version 1.2.
Better ADQL parsing.
Improvements and adjustments to
API to facilitate static acquisition of
report codes during programmatic use.
A few error codes have changed.
- Add new "duff query" test to
taplint MDQ stage data type mismatch error
report for BOOLEAN/boolean declared/returned data.
taplint now takes steps to ensure that TAP_SCHEMA
column list query is not truncated.
taplint now flags absence of ObsCore table with
I[nfo] not F[ailure] status.
- Change the implementation of
taplint stages which
perform validation against XSD schemas.
Schemas from external namespaces may now be imported and used.
The CPV stage, which was previously broken and disabled
by default, is now fixed and enabled by default.
Known/expected schemas are stored locally, and a warning is
reported if external ones are used. Schema validation seems
remarkably complicated, so it's possible there are still errors
in this implementation - if you suspect so,
please report it.
- Add missing geometric reserved words to ADQL reserved word
list. This fixes some problems with column names like
"DISTANCE" in taplint tests.
- Fixed some bugs related to TAP table uploads.
In particular these could cause incorrect table upload error
Version 3.0 (3 October 2014)
New plotting commands:
- A set of new plotting commands are provided which give
comprehensive access to all the new-style visualisation capabilities
available in TOPCAT v4. These commands are
documented in Section 8.
These commands, and the underlying visualisation facilities,
are considerably more capable than the, now deprecated,
old-style plot commands
- Programmatic invocation of STILTS tasks from third-party java code
is now officially sanctioned and documented in the new
To support this changes have been made to the parameter system
Parameter class now supports generics)
and there are some visible
changes to the user documentation as well
(parameters now report their data type,
and tasks report their classname).
Normal (e.g. command-line) usage should not undergo any changes,
but a fair bit of UI code has changed, so unexpected problems
- Add new output mode
which displays the table data in a scrollable
window on the screen.
- Add new
-allowunused flag to the stilts command.
If this is set, then unused parameter settings on the command line
just result in a warning, not failure of the command.
- Attempting to write FITS tables with >999 columns now fails
with a more helpful error message.
- Improved Unicode handling in VOTables.
Fixed a serious bug in
that generated unreadable output to BINARY or BINARY2 serialization
if any non-empty column had datatype="unicodeChar".
Also improved behaviour when copying between tables with
unicodeChar columns; these are usually preserved now, rather
than being squashed to datatype char.
Some lurking Unicode-related issues remain.
- The TAP client now tolerates whitespace around
UWS status codes.
- taplint: downgrade unknown post-table QUERY_STATUS value message
from Error to Warning.
Version 3.0-1 (13 November 2014)
- Add (experimental) read-only support for Gaia/DPAC GBIN format.
- Add new task
- Functions in class
nearMoc, and MOC can be identified by
VizieR table IDs as well as by filename/URL.
flags to control sequence of output rows.
allow use of an algebraic expression for values,
not just a literal value.
- Add special values
to refer to the column and row counts in a table.
The special variable
index is also deprecated in favour
Bugfixes and minor improvements:
- Add some more colour maps for aux/density shading.
stilts invocation script to pick up classes from
stilts.jar in script directory in preference to
other places (e.g. topcat-full.jar).
taplint to permit application/xml
not just text/xml content-type where appropriate (UWS stage).
taplint so it doesn't warn (W-TMV-UNSC) about
unknown VOSITables schema.
taplint so that
matches CHAR/VARCHAR in the same way as
for column type declaration purposes.
taplint so that capabilities document can have
TAPRegExt dataModel ivo-id elements with xs:anyURI rather than
vr:IdentifierURI (only a warning is issued in the latter case),
in anticipation of TAPRegExt-1.0 Erratum #1.
taplint to handle adql:TIMESTAMP columns more
carefully on upload and retrieval.
- Update JSAMP to v1.3.5.
Version 3.0-2 (6 February 2015)
- Linear fitting of points is now available using the
linearfit layer type for
plot2plane. Points may be weighted.
- You can add titles to plots using the new
- New plot layer type
allows plotting (optionally autoscaled)
markers with horizontal and vertical extents
independently determined by input data.
- More flexibility when assigning colour maps, in
density shading modes, and
*func allow assignment of
different data->ramp mapping functions
(sqrt and square as well as linear and logarithmic),
and new parameters
*quant allow quantisation
of the colour map to discrete levels.
maxsizeN parameter with
size plot layer type.
You can now optionally turn off autoscaling and specify marker size
in pixels instead.
auxcrowd parameter to plot2 tasks to influence
tick crowding on aux axis colour ramp. Also adjust default to use
- Add some "dart" options (fixed-base open or filled triangles)
for plotting vectors (see
arrowN parameter in
- Add some "triangle" options (variable-base open or filled triangles)
for plotting ellipses (see
ellipseN parameter in layers
- Histogram normalisation option adjusted
so that total area under bars,
rather than total height of bars, is fixed.
PlotDisplay class that forms the result of
plot2 commands can now have
registered on it. This lets you determine what point a user
has clicked on if you're using the plotting classes from
third party java code.
- Reworked part of the FITS table input implementation,
in particular adjusting the way memory mapping is done to reduce
resource requirements on some platforms.
If you notice any difference, it should be reduced virtual and
perhaps resident memory usage, and some (~10%?) performance
improvements, when reading large FITS/colfits files.
If you were previously having problems
with large memory allocations leading to
disk thrashing and system lockup when scanning files
larger than RAM (this didn't happen on all OSes),
these will hopefully have gone away.
However, please report anything that appears to be working worse
than before, or continued memory usage issues.
- Colfits files can now be accessed from streams,
not just uncompressed disk files
(though that's not necessarily a good idea).
Bugfixes and workarounds:
- Fixed a query bug (missing
in the multi-SSA mode (
This long-standing bug would have stopped this command working
at all with well-behaved SSA services.
- Fixed error in fits-var output
(PCOUNT header card did not include block alignment gap).
- Graphics coordinates are now calculated in floating point
rather than as integers. This fixes problems that could
cause scaled vectors, ellipses etc to be drawn with shapes
or orientations badly wrong due to rounding errors.
It also improves plotting of analytic functions, especially
to vector contexts (PDF/EPS).
- Fix some problems to do with zooming to very large/small
plot axis ranges.
- Hide error bars (etc) that would extend to negative values
on logarithmic axes; previously they were being drawn in anomalous
- Fix NullPointerException bug when null value was supplied to
multi-word parameter (e.g.
- Fix Aux axis positioning for 3D plots so that the numeric labels
don't get snipped off at top and bottom.
- Add a hack that allows LDAC FITS tables to be treated sensibly
in auto-format-detection mode.
- Make VOTable handling more robust against unknown (illegal)
- Add missing parameters
to plotting task documentation.
Version 3.0-3 (14 April 2015)
- New System Command option for
input table syntax;
you can now use "
<syscmd" or "
to supply input byte streams from Un*x pipelines.
- Add new Kernel Density Estimate plot layer types
- More histogram normalisation options provided.
Instead of just
parameter of the histogram layer
now has the options
This allows both the area normalisation introduced in v3.0-2,
and the height normalisation used in earlier versions
which it replaced.
- More histogram bar style options provided;
parameter now provides the options
semi_filled (the new default) and
These give outlined partially transparent bars, which make it
much easier to see what's going on in multi-dataset histograms.
semi_steps does not currently export very nicely
Similar options are also available in the new KDE plots.
- Column data read in as unsigned bytes will now be written out as
unsigned bytes where the output format permits;
previously they were forced to 16-bit signed integers.
This affects FITS, VOTable and CDF I/O handlers.
count_rows() method to JyStilts
table objects, which for non-random tables
may be much more efficient than
- Be less strict about recognising colfits files
(tolerate implicit TDIMn headers).
taplint is now aware of, and performs some checks
related to, schema-level table metadata declared by TAP services.
- Work round FITS read bug that could cause problems for VOTables using
inline FITS serialization, and possibly elsewhere.
- Fix bug that caused trouble when auto-ranging a plot
with a single sky position.
Version 3.0-4 (17 August 2015)
Bugfixes (some significant):
- Fix a serious bug in processing of FITS bit vector
Values read from these columns are presented as a
boolean array. In all previous versions of STIL
the bits have appeared in that array in the wrong sequence
(LSB..MSB per byte rather than the other way round).
Apologies to anyone who may have got incorrect science results
from this error in the past, and thanks to Paul Price for helping
to diagnose it.
- Fix a less serious bug with
attempting to read a single-element bit vector column
resulted in an error making the file unreadable.
Values read from such columns are now presented as
- Fix a VOTable reading bug relating to bit vector data
datatype="bit") appearing in BINARY/BINARY2
serializations. This one was more obvious, it would usually
generate an error when attempting to read the file.
- Fix serious bug in time conversion for CDF TIME_TT2000
- Fix a bug in
votcopy that converted columns from
transforming. Since v3.0-3 this is no longer necessary.
In the case of converting to a binary serialization,
since v3.0-3 this was causing it to generate unreadable VOTable
- Fix a bug in
votcopy that failed to handle
columns with datatype
bit. In the case of
converting to a binary serialization, these were in all previous
versions generating unreadable VOTable output.
Now they convert them to columns with datatype
boolean (not perfect, but better).
values were being used the wrong way round.
- Upgrade JEL to v2.0.2.
Fixes problem with evaluating void-typed expressions,
and possibly some other obscure bugs.
- Some taplint bug fixes.
- Changes to the way that TAP service table/column name reports are
interpreted (to conform to original intention of TAP standard).
Taplint now checks that table/column names from TAP_SCHEMA
endpoint are regular-or-delimited-identifiers, but no longer
submits example queries using supplied column names wrapped in
- Modify the heuristics that determine whether the first row of
a CSV file is a header.
Enhancements (mostly minor):
- Added some functions to the
that return array-valued results from array-valued parameters:
- Improve error reporting in the face of non-VOTable
TAP error responses.
In many cases this makes it much easier to see what's going wrong
with a TAP query.
- As a diagnostic tool, when making TAP queries,
a log message giving a roughly equivalent
command is now issued at the CONFIG level
(visible using flags
limits the number of tables tested in the stage that
queries data from each individual table (
May be useful for very large services.
to determine what VOTable serialization variant is used to
transmit uploaded tables to the TAP server.
Previously uploads were always BINARY which ought to work,
but the parameter now defaults to TABLEDATA,
since some services (e.g. CADC)
currently fail with binary uploads.
- Minor improvement to version reporting
(reports java specification version,
no longer issues warning for absent revision string).
- Update JCDF library to v1.1
(minor changes to do with leap seconds).
Version 3.0-5 (22 October 2015)
- Fix error reporting bug when a non-VOTable response is received
from a TAP service.
- Upgrade to JCDF v1.2 - fixes a bug when reading large (multi-2Gb)
- Added source code for an example basic GUI plot application,
- The expression language has a new way of
referring to a column;
if you use the form "
you get the value as an Object not a primitive.
This is a special-interest measure for user-defined
functions that need to see null numeric values.
- Adjust GBIN input handler:
avoid descending into Class-typed members of gbin list objects,
and add logging for object->column translations.
Version 3.0-6 (27 November 2015)
Crossmatching bug fix
- Fix a long-standing crossmatch bug
relating to range restriction during pre-processing.
This could have caused missed associations (but not false positives)
near the edge of coverage regions when using per-row errors,
if the scale of the errors differed
(especially differed significantly) between the matched tables.
matcher values of
Thanks to Grant Kennedy (IoA) for reporting this bug.
- Some more options for making weighted density plots have been added.
Since v3.0 the Density shading mode has let you see the density
of plotted points, but this lacked some features.
Three new ways to do density plots are added; these all give you
the option of weighting by an additional coordinate (like the Aux mode),
choosing the combination method (mean, median, sum, max, ...),
and displaying the quantitative value-colour mapping
on the shared colour ramp (previously aux axis) beside the plot.
The new density plots are:
The details are somewhat experimental and may
undergo some adjustments in future releases (feedback welcome).
Weighted shading mode,
using shaped marker kernels on the screen pixel grid,
available for all plot types
using HEALPix bins on the celestial sphere, for the Sky plot
- Density layer
(later obsoleted by Grid layer),
using square N*N screen pixel bins, for the Plane plot
- There are various changes affecting selection and display of
colour maps used for density and aux axis shading:
- The default colour map for Aux mode, and other layers using
the shared colour map, is no longer Rainbow! It's Inferno.
Rainbow colour maps are much hated by visualisation experts.
Of course you can still choose Rainbow if you like.
- Add some new colour maps:
Viridis, Inferno, Magma and
the SRON rainbow variant developed by
some diverging maps
(HotCold, RdBu, PiYG, BrBG)
and a qualitative constant chroma/luminance map
- The options for Density and Aux shading are now mostly the same
as each other except where there's good reason to differ.
Previously they were different in haphazard ways.
- An attempt is made to give the default form of each colour map
a sensible name, without leading minus signs.
- Fix it so that the whole range of each map is distinguishable
from white. This is a good idea when you're plotting symbols
on a white background, which is common in stilts.
Perhaps there are cases it's not such a good idea;
if you think so, complain and I may change it back.
- Try to fix it so that all the colour maps go in the same
direction (light->dark) where applicable.
- Throw out a couple of particularly useless colour maps.
- Colour map ramp display is now different for non-absolute maps;
their effect is shown on a selection of base colours,
not just for one base colour.
- Try harder to identify epoch columns (suitable for time plot),
in particular look for VOTable
xtype of JD or MJD,
units of year.
- Add some functions to the
to do with solid angles
- Fix plot bug; titles were painted in white
for pixel output formats.
- Rationalise plot report logging. Some more diagnostic information
about plots is now logged at the INFO level
(visible if topcat is run with the
Version 3.0-7 (10 June 2016)
- The JEL library underlying the expression language parser
has been upgraded to v2.1.1 (thanks to Konstantin Metlov),
now supporting variable-length argument lists among other things.
This allows the following improvements:
- New functions that support any number of arguments are provided:
in class Arrays;
in class Strings;
in new class
- Some old lists of similarly-named functions with fixed numbers
of arguments have been replaced by single functions that
take an arbitrary number of arguments
- The 2-argument
max functions in class
Arithmetic have been renamed
maxNaN to avoid confusion,
but in most cases existing expressions involving min/max
will work as before.
- Some functions that used to require string arguments will now
auto-convert numeric types
can now be written
- You can now implement
user-defined functions with variable
numbers of arguments.
- Writing large (>=2**31) literal integers used to fail with
an inscrutable error message.
Now the message tells you to append the
has been enhanced so that it can make multi-zone plots -
multiple plots stacked vertically that share the same horizontal
(time) axis but have independent vertical axes.
The time plot itself and this multi-zone feature are currently
experimental; in future versions they may be improved or changed,
and the multi-zone feature may be extended to other plot types.
Some other changes and fixes have gone along with this:
- A few API changes have been made to support multi-zone plots,
including generalising the
interface and rearranging some
constructor/factory method arguments.
For single-zone plots the changes are not very substantial.
This only affects you if you are using the STILTS classes
as a java plotting library.
If that applies to you and you have trouble upgrading,
I'm happy to provide assistance.
plot2time now supports shading modes
shadingN and associated parameters).
- The spectrogram layer
now uses the (per-plot, or more precisely now per-zone)
Aux colour map rather than a layer-specific colour map.
This means that the colour ramp is displayed alongside the plot,
but also that some parameters have been renamed
Z is an optional zone suffix
N is an optional layer suffix).
function layer type now works in the time plot,
rather than throwing an error.
However, it doesn't work very well, since the time coordinate
is in unix seconds rather than something more user-friendly.
- Fixed a serious bug in ISO-8601 axis labelling.
In some cases axis labels were being drawn at positions badly
different from the correct position.
Miscellaneous enhancements and changes
- This and subsequent releases target Java SE 6,
so will no longer
run under the (now very ancient) Java 5 runtime.
- Provide more careful documentation of licensing arrangements.
The distributed LICENSE.txt file notes that the starjava code is
LGPL, and documents licenses for each third-party dependency.
- Add Fill layer type for Plane and Time
tapresume now use
blocking HTTP requests rather than repeated polls
to wait for asynchronous TAP job completion from
services that declare themselves UWS 1.1 compliant.
- Add new parameters
These let you request resource limit adjustments when submitting
an asynchronous TAP job.
- Improve sky plot border painting.
- Clean up noisy Cubehelix colour map.
- New function
countTrue in class
- New stage
EXA for taplint
Note the details of the examples format are still under discussion,
(this version targets WD-DALI-1.1-20160415 &
WD-TAP-1.1-20160428, somewhat informed by TAPNotes-2013-12-13),
so the details may change in future.
taplint now validates ObsCore 1.1 where declared
alongside ObsCore 1.0.
Currently uses PR-ObsCore-v1.1-20160330.
- The taplint API
has changed slightly:
the class that used to be
is now called
If you are using taplint programmatically you may need to make
small changes. This results from some refactoring that makes
it easier to customise taplint output.
opaque config option with
transparency for plane and sky density plots.
- Changed implementation of GIF exporter for plots,
from Acme to ImageIO.
Shouldn't be any noticable difference.
Acme encoding dependency removed.
- Fix bug in cumulative histogram calculation.
- Fix read failure for FITS files with non-blank TDIM for
- Fix bugs that led to timezone-dependent results when
reading ISO-8601 or decimal year time columns.
- Fix numeric field truncation bug in LaTeX table output.
- Fix some parameter handling errors in
- Fix NullPointerException bug for disjoint regions in some cases in
Version 3.0-8 (13 September 2016)
- New task tskymap lets you generate all-sky
density maps (e.g. HEALPix) from an input table.
- New plot layer type healpix
can plot all-sky maps from HEALPix map tables
(e.g. as generated by
- Add some HEALPix-related functions to the
conversions from pixel index to sky position and
conversions between ring and nested schemes.
- Subrange-typed plot command parameters like
now have a default value of null.
This uses a default clip, which avoids very light colours.
Explicitly supplying a subrange value (e.g. "
can now use the whole range; previously the very light colours
- The GBIN input handler
can now pick up more metadata from the classpath.
For suitable tables, metadata included in datamodel classes
if present can be interrogated to provide table
and column descriptions and UCDs.
There are still some deficiencies of this functionality
(no column order, utypes and units missing,
large file "temp.xml" written to current directory)
dependent on issues in the upstream Gaia libraries and ICD.
- Add parameters
so you can specify custom locations for different TAP endpoints
rather than have them fixed at their default locations
hanging off the base service URL.
Ditto for the tasks
though in these cases the extra parameters are
(since currently rather special interest) largely undocumented.
taplint behaviour in absence of a /tables endpoint
(404 now gives a Warning not an Error).
taplint ObsTAP validation updated to
- Fix bug that caused read failures for large (>0.5Gb) FITS files
outside of the current directory on 32-bit JVMs.
This was a regression bug since v3.0-2.
- Fix long-standing bug that failed to release file descriptors
when opening FITS tables (could cause an error if very many
FITS files were opened).
Version 3.0-9 (23 September 2016)
- Fix aux ranging bug in some plot types
(SkyDensity, Healpix, Density, Weighted)
that used an aux data range too small to show colour variations
and negative data values were present.
- Improved performance (better memory use, faster) of some plot types
(SkyDensity, Healpix, Density, Weighted).
- Fix JyStilts bug that could cause the same data to be used for
different tables specified in a single plot.
Version 3.1 (8 March 2017)
- Improved documentation of plot
layer types and
shading modes in the user document -
each option now has an example graphic
and the text of the command that generates it.
- New layer type grid,
to plot a 2-d weighted histogram.
This replaces the
which has been withdrawn
grid can do all the same things and more,
except specify bin size in screen pixels).
- New plot layer type quantile,
which can (e.g.) plot median lines through noisy data.
- New plot layer type gaussian,
for Gaussian fits to histograms.
- Histogram-like layer types are now available from the
plot2time command as well as
so you can now plot histograms with a temporal horizontal axis.
- New normalisation (scaling) option
histogram, KDE, and KNN plots.
- New normalisation (scaling) options
for histogram, KDE and KNN layers when used from the
- Colour names recognised by the
now include the (140)
CSS-like colours alongside
the dozen standard plotting colours.
- Plot commands
insets parameter now lets you specify
margins round plots using any combination of
rather than requiring all values or none.
- Add new parameter
auxwidth to plot2 commands,
to control colour ramp lateral dimension in pixels.
- Modify plan caching arrangements for STILTS plots.
This results in much better interactive performance
(navigation etc) for some plot layer types plotted to the screen.
There are also some minor API changes to a few of the plot2
utility classes, which now provide more flexibility
- Various tweaks to the details of how plots are positioned on
the screen or in output graphics files.
- Fixed the match score (distance measure) for
Previously, the score was a linear sum of the unscaled distances
for the constituent matchers, which meant a Best match
was pretty meaningless.
Now, it adds scaled distances in quadrature,
so Best matching should give you a somewhat sensible result.
matchers now report output separations as scaled (dimensionless)
values rather than in physical units; this means the results are
more comparable, so Best matches will make more sense.
can now optionally write its output in JSON
taplint OBS stage for
- The Maths function
hypot now takes an arbitrary number of arguments
(instead of exactly two).
blockmaxrec parameter to
and improve warning text in case of result truncations.
when monitoring async jobs; if the service goes down,
the application will wait for it to come back rather than
- Options for MOC output format
in pixfoot command) have changed;
ascii is replaced by
The old ASCII format was slightly broken JSON in any case.
- MOC library upgraded to v4.6 (from v3.3). Improved MOC output.
- Update JCDF library to v1.2-2 (2017-01-01 leap second).
- Fix KDE/KNN plotting bug that could get scaling badly wrong
for normalised cumulative plots.
- Fix some regression bugs relating to
insets parameter, present since v3.0-6.
- Remove spurious padding from EPS graphics output.
- Fix small bug in
label painting; horizontal axis label was sometimes off the
bottom of the plot by a few pixels.
- Fix subpixel offset of colour ramp frame in PDF/PostScript graphics
Version 3.1-1 (29 September 2017)
- New plot layer types
skycorr for error ellipses rotated
as specified by Gaia-style correlation values.
- New plot layer type skygrid
can draw multiple sky system coordinate axis grids on sky plot.
- Colour map parameters
for the plotting commands
will now accept custom colour maps that interpolate between
a list of named colours,
- New position angle calculation functions
added to expression language.
Table I/O changes
- It is now possible to write and re-read tables with >999
columns to FITS or colfits format. The new limit is 2^31 columns.
This uses a non-standard convention; software that is not aware
of the convention (e.g. CFITSIO or earlier STILTS versions)
will only be able to use the first 998 columns of tables written
in this way.
- For VOTable columns that reference
the relevant information is now accessible as
column auxiliary metadata
e.g. using the
- Any columns referencing
read from VOTable-based formats (VOTable or FITS-plus)
will now be written out to VOTable-based formats with
equivalent COOSYS references included.
Currently not table parameters though.
- The default version for output VOTables is now VOTable 1.3.
New output formats
votable-binary2-href are now offered alongside
the five previously available VOTable variants.
- Slight changes to the FITS-plus output handler VOTable formatting
in the primary HDU; now uses default output VOTable version
rather than VOTable 1.1.
- FITS keywords using the ESO
convention can now be read as table parameters
rather than ignored when reading FITS tables.
Minor behaviour changes
charset parameter of
now defaults to UTF-8 rather than the system default.
- Replace parameter
- Change autoscaling defaults for plot layer types
autoscaling is now turned off by default.
Apologies for this change to behaviour,
but it's better for consistency with new layer types
and presents less danger of misinterpreted plots.
- Taplint: downgrade type mismatch error
W_QTYP in view of TAP-1.0 Erratum #3.
COOSYS element is now passed without a deprecation
warning in VOTable 1.3 documents by
in view of VOTable-1.3 Erratum #1.
The warning is still issued for VOTable 1.2 documents,
but the text is toned down.
- Taplint no longer issues a warning about DataModel references
that do not match the
this follows TAPRegExt Erratum #1.
- Modify taplint Examples document @vocab attribute testing in the
light of DALI 1.1.
- Update PixTools (HEALPix) library to 2017-09-06 version
This fixes a bug in
healpixRingIndex that could give
the wrong value for small values of longitude near zero
in the equatorial region.
It seems possible that this might have led to very infrequent
missed associations when crossmatching in these regions,
but tests appear to indicate that no such errors would
actually have resulted.
- Long fields (>10240 characters) in output CSV files
are no longer truncated.
- Fix misfeature in
skyellipse plot layers;
these now preserve orientation on the sky even when
viewsys differs from
parameters were always interpreted in
the view, rather than the data, sky coordinate system.
- Fix bug in plot title placement.
- Fix filename generation error for plotting command
animation output (wrong number of digits if frame count
was an exact power of 10).
Version 3.1-2 (7 November 2017)
New expression language functions
- New functions
for extracting positions from IAU-style object designations
2MASS J04355524+1630331) - use with care.
- New functions
these calculate the distance in 3d space between two
positions specified in spherical polar coordinates.
- New functions
to help with phase folding given a known period.
A new modulus function
whose output is always positive
% operator) is also added.
Bug fixes and workarounds
- Upgrade JEL to v2.1.2: now you can use functions in the expression
language that have the same name as table columns.
- Add fix to discard on read any VOTable/FITS-plus table parameters
with names "
These useless items were added in potentially large numbers when
saving plotted tables from TOPCAT v4.5 (TOPCAT bug).
tpipe operation at this version will therefore
purge these items.
- Fix plot bug that sometimes caused error bars to come out
very small when exporting to a graphics file.
This was a regression bug (present in v3.1-1 but not v3.1).
- Fix another tiny error bar plotting bug too.
- Small taplint update; required UCD for
to match REC-ObsCore-1.1 (even though it's an illegal UCD1+).
Version 3.1-3 (24 April 2018)
- New DataLink validator task
- Add class Gaia
to the expression language.
This contains functions for estimating distances from parallaxes
and propagating astrometric parameters and errors to
different epochs. The functions are not specific
to data from the Gaia astrometry satellite,
but are presented in a form convenient for use
with the Gaia DR2 source catalogue.
- New array manipulation functions
pick added to class
- New utility function
square added to class
for defining array and string shape/extent column metadata.
- DataLink-style service descriptor RESOURCEs in input VOTables
(and FITS-plus tables) now appear as table parameters.
- Various improvements to the Contour
You can now contour quantities weighted by a given coordinate,
rather than just point density.
The smoothing from weighted point samples to the grid being
contoured can be performed using sum, mean and
other combination methods,
it now uses a Gaussian kernel rather than a square top hat,
and performance is considerably improved especially at large
The contour levels used are now reported,
so can be seen by specifying the
The contours are now plotted correctly right up to the edge of
the visible plot, and various bugs have been fixed.
- The histogram plotter
now has a
for weighted histograms.
This means that you can plot e.g. the mean value of a given quantity
per interval on the X axis rather than just the summed quantity.
A corresponding (though somewhat less well-defined) option is
also provided for the
combine parameter that configures how
values are binned in various histogram-like plots
and since this version also
now has two new options,
These work like the existing
options, but scale the combined values by the relevant unit
(e.g. X axis unit or solid angle).
Where these units are physical, a
parameter is also provided for scaling in convenient units:
second, day, year etc for time
square degree, arcminute, arcecond etc for solid angle
In some cases, the
combine default values
- The same new
are also added to the tskymap command.
- The linearfit layer type
is now available for
as well as
Minor behaviour changes
votlint no longer considers an integer- or array-typed
PARAM element with attribute
to be an error.
votlint no longer Warns about
since this usage pattern is now found in
DataLink Service Descriptors.
- The IPAC table reader now matches data type specifications
- Improvements in documentation of the expression language functions:
in the Function documentation section
classes are now listed in alphabetical order,
and examples are included in some cases.
Some readability improvements have also been made in the function
browser displayed by the
taplint's VOTable validation now includes
validation as well as the custom VOTable checks.
It was always supposed to do this, but it seems that it didn't,
at least in recent versions.
- Update JCDF to v1.2-3; fixes some CDF reading bugs.
coneskymatch from sending cone search queries
- Minor fix in
taplint OBS stage;
s_region UCD is
(a relic from PR-ObsCore-1.1-20161004).
Version 3.1-4 (18 May 2018)
- Bugfixes and minor enhancements:
- Add new option
which lets you delete a TAP async job.
Also add a message to
inviting the user to delete interrupted jobs in this way.
- Upgrade Grégory Mantelet's ADQL parsing library to v1.4.
- VOTable output tweaked: single quotes now used for attribute
values if attribute contains double quotes.
- Fix bug in
caused a result read failure for result VOTables longer than 2Gb.
- Fix range bug; caused plot failure when plotting very
large values with no variation
(e.g. gaia_source solution_id).
Version 3.1-5 (2 November 2018)
- New spheregrid plot layer,
for plotting a spherical net around the origin,
- New line3d plot layer,
for plotting lines joining points in 3d,
for joining out-of-sequence points,
added to line plot layer.
- Replace skygrid plot layer
so you can control grid line spacing for meridians
and parallels independently. Note this is a backwardly-incompatible
change, so could break existing scripts.
This change may have introduced slight changes to sky axis
grid line spacing at low crowding levels.
Contour level calculations improved;
in some cases this previously didn't work well
at low point density, resulting in missing contours.
The meaning of the
zero parameter has also changed
slightly, it now defaults to 1 and is not phase folded.
- Sphere plot now centers axes on zero
if the center would otherwise be near zero.
- Address long-standing plot auto-ranging issue;
when auto-ranging resulted in the bottom X limit or
right-hand Y limit being exactly equal to zero,
the corresponding zero-valued results were not plotted.
Auto-ranging has been slightly adjusted to avoid that.
This results in pixel-level changes to plot appearance
in some cases; these can be avoided by explicitly setting
- New functions in class
These can calculate Cartesian position and velocity components
from (e.g. Gaia) astrometric parameters.
- New Shapes
class added for working with polygons in the X,Y plane;
- Rearrange expression language documentation slightly:
there is a new section listing
This includes "
$random", which was previously
undocumented and named "
- New functions
added to class
- New convenience function
exp10 in class
- Improvements to
taplint UWS stage:
now validates job documents
against UWS schema, tests
attribute, and does improved and version-sensitive
(UWS v1.0/v1.1) validation of UWS timestamps.
Also does more complete deletion of submitted jobs.
taplint schema validation now takes account of
VOTable 1.3 Erratum #2 -
FIELD/@precision attribute values
E0" are now permitted.
- The VOTable validator in
taplint no longer complains about multiple
INFO elements in the same scope sharing the
- Minor improvements to
to convert a run of adjacent scalar columns into an
array column (the opposite of
Minor enhancements and behaviour changes:
- GBIN read fix to work around changed behaviour in recent
GaiaTools (versions 19.4.*, >=20.1.0 and >=21.0.0)
that caused GBIN table reading to fail.
- Fix some additional commands
so they follow HTTP 3xx redirects automatically in more cases
when given a URL for input.
Most STILTS commands already do this.
- Introduce a couple of measures to reduce the likelihood
of unintentional service overload from
coneskymatch and friends:
add a progressively increasing delay for
error handling modes,
and decrease the soft maximum for the
parameter to 5 (from 10).
- Slight improvements to the
section of this manual.
- Small change to MOC handling that might possibly avoid some
network-related performance issues.
function plot layer so that NaN values are
omitted rather than interpolated.
stilts startup script now correctly follows
symlinks on some OSes where it didn't work before
(including OSX). It also looks for jar files to use in
slightly different places on OS X.
Version 3.1-6 (9 May 2019)
- Introduced new colour scaling options
histolog for the parameters
and the other plotting commands.
This can make it much easier to see structure in quantities
that do not vary smoothly over their min-max range.
- New plot layer types
introduced for drawing outline or filled quadrilaterals and
other polygons in all plot types.
line plot layer
now has an
aux parameter that can vary the colour
of the line along its length according to some
plot2sky now by default
draws a small scale bar at the bottom left corner of the plot,
indicating the scale in degrees, minutes or seconds.
It can be switched off using the new
- New option
Car0 for the
this is like
Car (Plate Carrée)
but has longitude=0 at the left/right edge
rather than the center of the plot.
- The healpix plot layer can now
plot HEALPix levels up to 20 (previously the maximum was 13).
Also, autoranging now works, so if a plot is made of a HEALPix
file covering part of the sky, the sky view is centered on it
in the same way as for marker plots.
Memory management is improved for fairly large maps.
- Various changes introduced to support the semi-standard
Available information about HEALPix encoding
(level, index column, ordering scheme, coord sys)
can now be stored in custom table parameters
(of the form
and is used by FITS output handlers to insert
the relevant FITS headers.
The existing FITS handlers do this where it's not disruptive,
and the new fits-healpix
output handler will additionally move and rename columns
This metadata is round-tripped by FITS and VOTable I/O handlers.
It is added automatically by the tskymap
command, and can be manipulated by hand using
the new healpixmeta filter.
FITS support is not perfect: the
FITS keyword is ignored, and the 1024-element array-valued
column variant is not understood.
VOTable 1.4 support
- Experimental support has been introduced for version 1.4
of the VOTable format and its new
This support corresponds to WD-VOTable-1.4-20190403,
with the correction that the RDF vocabularies for TIMESYS
have URIs using the authority "www.ivoa.net" and not "ivoa.net".
The functionality may be modified in future releases depending
on how the VOTable 1.4 specification evolves.
This VOTable 1.4 support has resulted in some minor related
now supports Version 1.4 VOTables:
the 1.4 draft schema is used for XSD validation,
and the TIMESYS element is checked
for attribute content and suitable referencing.
- For VOTable columns that reference TIMESYS elements,
the relevant information is now accessible as
column auxiliary metadata
e.g. using the
- Any columns referencing TIMESYS elements
read from VOTable-based formats (VOTable or FITS-plus)
can now be written out to VOTable-based formats with
equivalent TIMESYS references included,
so TIMESYS round-tripping for columns works;
however this will only be done if the VOTable output
format is set to version 1.4.
By default (at least as long as 1.4 is not finalised)
the output version is 1.3.
To enable this TIMESYS output, set the
Currently this TIMESYS output works only for table columns
(FIELDs) not parameters (PARAMs).
timeoffset attribute of a TIMESYS element
referenced by a VOTable column is used to make sense of
column data when interpreting it as an absolute time.
Currently, the only use of this is in
FIELD/@ref attributes are no longer imported
VOTable ref" column aux metadata items,
since they often interfere with TIMESYS references.
Doing this was probably always a bad idea since the referencing
is not kept track of within the application, so withdrawing
this functionality makes sense, but beware that it might
change or break some existing behaviour.
votlint modifications above may now
interrogate external vocabulary resources during validation,
meaning that external network connections may be made during
validation, which didn't happen in previous versions.
- New functions
indexOf in class
to find position of a given value in an array.
- New functions
for extracting array values from strings (experimental).
- Fix expression compilation so that
$ID column references
referring to nonexistent columns are rejected at compile time
rather than causing trouble during evaluation.
- The validator tasks
insert the token "
(IVOA-validate)" into the
User-Agent header of all HTTP requests they make,
for convenience of services that want to identify validators.
This is a convention discussed within the IVOA Operations IG.
Bug and misfeature fixes
- Improve colour ramp quantisation
the full color range is now included.
- Use period not comma as decimal separator for non-sexagesimal
plot2sky axis labels regardless of Locale;
also avoid trailing comma sometimes erroneously present.
- Fix bug/misfeature in CDF table parameter construction:
CDF global attributes were ignored
(with a "WARNING: Omitting complicated global attribute" message)
if they contained any null entries.
Now such entries are just ignored and the table parameter is
constructed from the global attribute using the
- Fix some bugs relating to plotting values close to the limits
of the double precision range.
- Fix a problem with the link2
plot layers on sky plots with Aitoff/Car projections
that caused short lines that should span the antimeridian
to appear as long lines crossing the whole sky.
Such links are now just not drawn.
- Fix misleading error message about
for underconfigured plot layers in
Next Previous Up Contents
Up: Release Notes
STILTS - Starlink Tables Infrastructure Library Tool Set
Starlink User Note256
STILTS web page: