Releases to date have been as follows:
stilts",
invoked using the stilts script or the
stilts.jar jar file, and the various tasks are
named as subsequent arguments on the command line.
Command arguments are supplied after that.
The new invocation syntax is described in detail elsewhere in
this document. As well as invocation features such
as improved on-line help, optional prompting,
parameter defaulting, and more uniform access to common features,
this will make it more straightforward to wrap these tasks
for use in non-command-line environments, such as behind a
SOAP or CORBA interface, or in a CEA-like execution environment.
tmatch2 has been introduced.
This provides flexible and efficient crossmatching between
two input tables. Future releases will provide commands for
intra-table and multi-table matching.
tcat has been introduced, which
allows two tables to be glued together top-to-bottom.
This is currently working but very rudimentary - improvements
will be forthcoming in future releases.
calc has been introduced,
which performs one-line expression evaluations from the
command line.
tpipe
and other commands have been introduced:
addskycoords: calculates new
celestial coordinate pair from existing ones
(FK4, FK5, ecliptic, galactic, supergalactic)replacecol: replaces column data,
using existing metadatabadval: replaces given 'magic'
value with nullreplaceval: replaces given 'magic'
value with any specified valuetablename: edits table nameexplodecols and explodecols commands
replace explode
The new stream parameter of tpipe now
allows you to write filter commands in an external file, to
facilitate more manageable command lines.
Wildarding for column specification is now allowed for some filter commands.
tcube Command
stats filter provides the same information as
the old stats output mode, but allows much more
flexible use of the results. It can also calculate many new
quantities, including quantiles, skew and kurtosis.meta filter provides the same information as
the old meta output mode, but allows much more
flexible use of the results.assert filter provides in-pipeline logical
assertions.uniq filter collapses multiple adjacent identical
or similar rows.sorthead filter provides a (usually) more
efficient method of doing what you could previously do
by combining sort and head filters.colmeta filter adds/modifies metadata for selected
columns.check filter checks table in stream - for debugging
purposes only.Additionally usage of the sort filter has been changed
so that it can now do everything that sortexpr used to
be able to do; sortexpr is now withdrawn.
plastic mode broadcasts the table to
one or all registered PLASTIC listeners.cgi mode writes the table to standard output in a
form suitable for output from a CGI script.discard mode throws away the table.topcat mode now attempts to use PLASTIC
(amongst other methods) to contact TOPCAT.stats and meta modes are mildly
deprecated in favour of the corresponding new filters
(see above).csv-noheader format variant output handler
added.roundDecimal and formatDecimal
functions introduced for more control over visual appearance
of numeric values.mark.workaround
system property which can
optionally work around a bug in some input streams
("Resetting to invalid mark" errors).ucd and
utype attributes of TABLE element in
votlint.istream=true is now less likely to cause a
"Can't re-read stream" error.colfits format, new startable.storage
policy "sideways") have been introduced.
These can provide considerable efficiency improvements for
certain tasks when working with very large (and especially wide)
tables.
multicone
- Makes multiple cone search queries to the same serviceregquery
- Queries the VO registrytranspose filter added.setparam and clearparams filters.addskycoords.tcat command has been replaced by more capable
tcat and
tcatn commands.
Between them these provide concatenation of an unlimited number of
homogeneous or heterogeneous input tables.
Additional columns may be added to indicate which of the input tables
given output rows originated from.
in in tcat,
cmd and friends)
may now be specified in the form "@filename".
This indicates that the value for the parameter is to be obtained
by reading it from the named file.
This is useful if a very long value is required for the parameter
in question. The script parameter of
tpipe has therefore been
withdrawn, since it did just the same thing.
ivo:- or myspace:-type URLs.
toHex and fromHex numeric conversion
functions have been added
(Section 9.5.4).commands etc.-J flag to stilts script
for passing flags directly to Java.out parameter to votlint.-ifndim and -ifshape flags to
explodeall filter.exact match mode in
tmatch2
now copes with array-valued columns.force parameter to multicone task
as a workaround for some broken services.stats filter.votlint about
utype attribute on RESOURCE elements.tjoin introduced.votlint
and votcopy.-bench flag to stilts command.text and ascii
output formats (now one-pass not two-pass).duptag parameters to tmatch2
task for customised renaming of columns with duplicated names.sinh, cosh, tanh
and inverses).funcs task, a browser for
expression language function documentation.-checkversion to list of stilts
flags.table parameter to
calc command
(for access to table parameters).param$ notation (Section 9.2).ucd$ notation
in expressions (Section 9.1)
and as column identifiers (Section 6.2).TNULLn header
cards - write them as numeric not string values.gcj).formatDecimalLocal functions in
class Formats.fluxToLuminosity and luminosityToFlux
functions in class
Fluxes.transpose filter.sqlcone
task introduced, along with some classes in package
uk.ac.starlink.ttools.cone designed for library use by
AstroGrid DSA code.
tmatch2 params
parameter now has minoccurs=0, since that can be true
for exact matches.-stdout and -stderr flags to
stilts command.app-description.xml
file.sqlcone command.fluxToLuminosity function.tmatch2
command are new tasks:
tskymatch2:
stripped down version of tmatch2 for ease of use when
matching with sky coordinates.tmatch1:
internal matcher, finds groups of objects within a table.tmatchn:
finds group or multiple-pair matches between multiple (>2)
tables.Two tasks have been renamed for improved clarity and consistency:
multicone is now named
coneskymatch
sqlcone is now named
sqlskymatch
There has also been some enhancement and rationalisation of
parameters for all table join tools (tmatch* as well as
tjoin,
coneskymatch and sqlskymatch):
fixcols and
suffix* parameters to control renaming of duplicated
columns in output tables (note this replaces the old
duptag* parameters in tmatch2).progress parameter
which allows you to configure whether progress is reported to the
console.copycols parameter of coneskymatch
and sqlskymatch now defaults to "*"
(include all columns from input table in the output).Section 7 of the manual has been somewhat rearranged and improved.
votlint handling of TABLEDATA-type
multi-dimensional char/unicodeChar arrays.
These are now split up into strings by counting characters rather
than using whitespace delimiters.
I think it's doing the right thing now.sqlclient,
which is a general JDBC-based SQL command-line client.sqlupdate,
which allows updates to existing rows in SQL tables.tosql
output mode:
write
(can be create, dropcreate
or append)newtable renamed
dbtable
database renamed
db for consistency with other commandsscorecol added to
tmatch2,
coneskymatch and
sqlskymatch commands,
which controls adding a new column to match output tables
containing a goodness-of-match value.parallel added to
coneskymatch task
which allows multiple cone searches to be carried out in
parallel.erract added to
coneskymatch which controls response to
isolated failures in individual cone search queries.-debug flag).help='*' which prints help for
all parameters of a task at once.+verbose flag
for reducing verbosity level.datatype attribute.progress parameter to tmatchn.emptyok parameter to
coneskymatch.calc task.stats filter cardinality value
calculation.fixcols.stats filter
usage message.server is
provided which allows STILTS commands to be executed via HTTP.
One purpose of this is to facilitate server-side use of the
plotting commands co-located with data to generate on-the-fly
graphical summaries of server-held datasets.
minReal and maxReal functions
(max/min ignoring blank values) in class
Arithmetic.
tcat's
loccol parameter.utype$ notation
in expressions (Section 9.1)
and as column identifiers (Section 6.2).regquery command has changed in implementation,
data access, and output format. It now queries VOResource1.0
registries rather than the very out of date registry protocol which
was used in earlier versions.fits-basic output
handler as signed byte values (TFORM=B,TZERO=-128)
rather than signed shorts (TFORM=I).regquery)
to fail for Java 1.6.votlint's validation tests
(VOTABLE element content model,
INFO and PARAM and FIELD required attributes).TFORMnn=rAw)
is now understood for FITS binary tables.addskycoords filter).votcopy: XML processing instructions
garbled on output, and pathnames in base parameters
inappropriately flattened in hrefs attribute values.votlint can now validate VOTable documents following
the (provisional, 2009-09-29 PR) VOTable 1.2 standard.votable.namespacing system
property.votlint now checks that the correct XML namespaces are
in use.parse* string->numeric conversion
functions now cope with leading or trailing whitespace.coneskymatch can now match using SIA and SSA
services as alternatives to Cone Search ones
(see its new servicetype parameter).startable.storage policy "adaptive"
is now the default. This should mean running out of memory less
often. The old behaviour can be restored by giving the new
-memory command line flag.fixcolnames.coneskymatch to fail in locales that use ","
for a decimal point.tmatch1, tmatch2, tmatchn and
tskymatch2).
For several common regimes, using default settings,
memory use has been decreased by a
factor of about 5, and CPU time reduced by a factor of about 3.tuning for
tmatch1, tmatch2 and tmatchn,
and parameter healpixk for tskymatch2).
Experimentation with these can lead to significant performance
improvements for given matches.find=best"
match assignments when pair matching in crowded fields.
Crossmatch results thus may differ between earlier versions
and this one. Both are reasonable, but the newer behaviour is
more correct. In non-crowded fields, there should be no change.malloc())
for intermediate-sized buffers
to avoid running out of java heap space.find=each" for
coneskymatch and sqlskymatch commands.
This allows you to get an output table with exactly one row for
each row of the input table.-memgui to monitor memory usage during runs.rowrange.array functions for constructing arrays,
and new aggregating functions median and
quantile.progress parameter
has changed; it now has an additional option which will write
limited profiling information as well as logging as the match
progresses.ylabel parameter to plothist command.random and sequential filters have
been renamed randomview and seqview
respectively. This provides a better idea of what they do.
Since they are only useful for debugging, it is unlikely that this
will break anyone's existing code.random introduced which converts tables
to random-access if necessary.legend parameter to
plotting commands.matcher parameters can now accept
classnames of MatchEngine implementation classes as
an option.stilts_jars.zip) as an alternative to
the monolithic jar file (stilts.jar).
This may be more appropriate for those using STILTS classes in a
framework that contains other third party class libraries.tmulti and tmultin.
These currently just copy multiple input tables to a single
multi-table container file (e.g. Multi-Extension FITS or
multi-TABLE VOTable). Future releases may generalise the output of
multi-table processing.multi parameter introduced for tcat
and tmulti tasks to pick up all tables in a
multi-table container file.treads and twrites
for multi-table I/O.addresolve;
this currently uses Sesame.repeat, which repeats table rows a
given number of times.long" in IPAC format
tables.meta filter.Finally, from this release STILTS requires version 1.5 (a.k.a. 5.0) of the Java J2SE Runtime Environment; it will no longer run on version 1.4, which is now very old. I don't expect this to cause compatibility issues for anyone, but I'm interested to hear if that's not the case.
http://user:pass@host/path) on
table URLs handled.tapquery and
tapresume have been introduced.
These provide support for the Table Access Protocol (TAP),
and allow freeform queries in an SQL-like language to be
made to remote databases.
soapout parameter to regquery
command.count, variance and
stdev functions to
Arrays.funcs window display.replaceval filter to work with Infinities.dashNS and
linewidthNS to plot2d task.taplint.
This is a validator for TAP (Table Access Protocol) services.
It is only likely to be useful to people developing or operating
TAP services.nan and inf representations.Infinity and NaN.coneskymatch cone search verbosity parameter so that
VERB=3 is not erroneously ignored.best1
and best2 have been added
for the find parameter in the pair matching commands
tmatch2 and tskymatch2.
They correspond to finding the best match in table B for each row
in table A, and in crowded fields often provide more intuitive
semantics than the previous symmetric best option
(in non-crowded fields there is generally no difference).
This replicates the matching performed by some other tools,
including Aladin.2d_ellipse,
and
skyellipse.d_err.
skyerr
matcher have changed slightly.Coords class has been replaced by
CoordsDegrees and
CoordsRadians
classes providing sky coordinate functions,
and a new class
TrigDegrees
provides normal degree-based trigonometric functions
alongside the radian-based versions in
Maths.
Some of the old function names have changed to make clear that they
use radians and not degrees.
This change should be much more convenient in most cases;
sorry it's taken so long to get round to.join function is added to the
Arrays class
to combine all the elements of an array into a string.taplint, mostly thanks to bug
reports etc from the TAP community:
ivo://ivoa.net/std/TAPRegExt#upload-*)
as per most recent TAPRegExt draft.parse to tapquery command,
allowing pre-send syntax checking of submitted ADQL.star.basicauth.user and
star.basicauth.password.coneskymatch in the presence of
unreliable or inconsistent DAL services.pixsample
which can sample pixel data from HEALPix table files
(useful for things like Schlegel dust extinction).
Also addpixsample filter,
which does the same job.pixfoot
which can generate MOC (Multi-Order Coverage) maps.coneskymatch
when using some Cone Search services (mostly VizieR).
This uses the Multi-Order Coverage map service operated by CDS.
It can make VizieR multi-cone queries much faster by not doing
cone searches that are outside the coverage region of the
catalogue in question.inMoc).votable.version.
Output version is VOTable 1.2 by default.votlint has been changed so that it handles
different VOTable versions more capably.
Versions 1.1+ are now validated against a schema
(which is how those versions are defined) rather than against
a DTD hacked to do the same job as the schema.
VOTable 1.3 validation is now provided.votcopy command has a new version
parameter to control output version,
and a new nomagic parameter to control whether
VALUES/null attributes are removed
where appropriate.+Inf"/"-Inf",
not "Infinity"/"-Infinity" as in
previous versions).votlint is now stricter about floating point
TD element contents.schemaLocation
attribute by default.hypot (=sqrt(x*x+y*y))
to the Maths class in expression language.split functions for string splitting
to the Strings class in expression language.-utype flags for addcol,
replacecol, colmeta and
setparam filters,
and utype option for meta filter.toString function:
it now works on non-numeric values,
gives the right answer for Long integers
and character values,
and returns a blank value rather than the string "null" or "NaN"
for blank inputs.OBS stage (ObsTAP validation)
to taplint.CAP stage of taplint.
Declared languages (including features) and output formats are
now checked.coneskymatch
parallel parameter.tmatchn group mode which could result in
output rows with columns from only a single table,
i.e. not representing an inter-table match,
even when join*=default.TFORMn=rC/rM).replacecol and replaceval
filters which could cause truncation of strings in FITS
and possibly VOTable output when the new value was longer
than the previously declared maximum length.tcat, tcatn so that in most cases
output column metadata is compatible with all input tables, not
just the first one in terms of nullability, array shape etc.rows"/"bytes"
rather than "row"/"byte"
for TAP capability unit values.