Q1.1 What is TOPCAT?
TOPCAT is an interactive graphical tool for analysis and manipulation of tabular data. Its manifesto is:
Does what you want with tables
Q1.2 What is a table?
A table, for TOPCAT's purposes, is a data structure with columns, so that all the values in a given column are of the same kind. The types of value in each column are usually, but not necessarily, numbers, strings, or arrays of these. The most important example in astronomy of a table is an object or source catalogue. Two- or multi-dimensional arrays, such as images or data cubes are not tables - TOPCAT will not be able to help you with these.
Q1.3 What are TOPCAT's strengths?
Q1.4 What are its weaknesses?
Some of these capabilities (limited scriptability, large tables) are however provided by TOPCAT's sister package, STILTS.
Q1.5 Is TOPCAT just for astronomy?
The large majority of TOPCAT's user base is currently in astronomy. However many of its facilities are not astronomy-specific; it is quite suitable for use in other contexts, and has been so used. I'm very open to approaches from other scientific domains who are interested in using it, and possibly customising it accordingly, but the amount of effort to publicise the software effectively in another user community would be considerable, and I'm not currently doing that.
Q1.6 Is TOPCAT just for the Virtual Observatory?
No. In many ways, TOPCAT is a traditional data analysis tool, in that it will perform most of its functions quite happily with no network connection. It does however provide a number of VO-friendly facilities.
Q1.7 Why is TOPCAT written in Java?
Several reasons, some of the important ones being:
Q1.8 But isn't Java slow?
Q1.9 Gosh what a terrific bit of software. How can I express my appreciation?
Why thank you. The best thing you can do is to help publicise its use by telling your colleagues how you've found it useful. Acknowledging its use in published papers and talks is also very helpful. Telling me what you've used it for can (a) give me a warm fuzzy feeling inside and (b) come in useful when it comes to justifying development effort, e.g. to funding agencies.
Q1.10 How should I acknowledge TOPCAT?
It is much appreciated if you can acknowledge TOPCAT in your publications and presentations. The preferred method is to cite one or both of these papers:
Alternatively, you can cite one or both of the following permanent URLs:
I keep an eye on the ADS links above for citations of these papers. If you publish anything that cites or otherwise uses this software outside of the normal astronomy literature, I'd be most interested to hear about it.
Q1.11 What other papers are there relating to TOPCAT?
The following papers relate to TOPCAT and STILTS development and functionality:
2005ASPC..347...29TTOPCAT & STIL: Starlink Table/VOTable Processing Software
2006ASPC..351..666TSTILTS - A Package for Command-Line Processing of Tabular Data
2008ASPC..394..422TColumn-Oriented Table Access Using STIL: Fast Analysis of Very Large Tables
2009ASPC..411..510TSTILTS Plotting Tools: Scriptable graphics generation from small and large tables
2014ASPC..485..257TVisualising Large Datasets in TOPCAT v4
2015ASPC..495..177TExternal Use of TOPCAT's Plotting Library
2017ASPC..512..589TTOPCAT's TAP Client
10.3390/informatics4030018TOPCAT: Desktop Exploration of Tabular Data for Astronomy and Beyond
2020ASPC..522...67TTOPCAT: Working with Data and Working with Users
2019ASPC..523...43TTOPCAT and Gaia
Q1.12 Where can I find old/new versions?
The most recent public release is available from the web page. Previous versions can be found in the release archive. New releases are notified on both the mailing lists.
There will sometimes be a bleeding-edge version in the pre-release directory - this may contain fixes and new features not yet available in the public release which you can experiment with at your own risk.
Q1.13 How do I get the source code to look at?
The source code for TOPCAT and friends is available in the Starlink/starjava repository on Github.
If you just want all the source code just to look at,
an alternative may be to grab the
topcat_src.zip file from the web page,
which contains all the java source code for TOPCAT, STILTS and the
uk.ac.starlink.* classes it uses.
Third party library code is not included.
See also Q1.14.
Q1.14 How can I compile TOPCAT from source code?
The proper way to build the application from scratch is to check out the starjava build system from the starjava git repository. No password is required for read access. Do something like:
% git clone git://github.com/Starlink/starjava.git starjava/sourceYou can then read the file
starjava/source/READMEfor instructions on what to do next. Your mileage may vary somewhat, but the sequence will be along these lines:
% cd starjava/source % setenv STAR_JAVA /usr/java/jdk1.8.0/bin/java .. or wherever % cd ant % bin/ant build install % cd .. % ../bin/ant build % ../bin/ant installTo build the standalone TOPCAT jar files
topcat-lite.jaryou can do:
% cd topcat % ../../bin/ant build-standalone installYour JDK may be any version >=1.6, and must have JAI (Java Advanced Imaging) installed. Note that this process builds the whole starjava suite of software, which includes more than just TOPCAT.
Since the standalone jar file
contains all the library classes, a quick and dirty alternative
is to compile from the sources in the
topcat_src.zip file from the web page
topcat-full.jar on your classpath at compile time.
Q1.15 I think I've found a bug.
If there's something wrong which is not noted in Section 4, please mail me and let me know. I may be able to give you a workaround or updated copy, and more importantly it gives me a chance to fix it in future versions.
Q1.16 Why doesn't TOPCAT display interactive sky images/show movies/speak my weight?
I have a long list of features that I'd like to add eventually, some of which will probably get done and some of which will probably not. However, I'm always interested in new suggestions. The existing feature set is very largely due to suggestions and requests from existing users.
Having said that, there are some things which I am unlikely to add, in particular things which are not basically to do with manipulation of tables, and things which would require non-Java code to be introduced.
Q1.17 How can I find out when new versions are released?
Subscribe to the
topcat-announce mailing list - see
list information here.
Q1.18 What license is TOPCAT released under?
The application is released under the GNU General Public Licence (GPL). However, many of the underlying libraries are available under more permissive licenses such as the LGPL. There is some more information in the STILTS LICENSE.txt file.
Q1.19 What related software is there?
The following items are related to TOPCAT:
Q1.20 Who has developed TOPCAT?
Most of the code has been written by Mark Taylor. Much help has been supplied by other members of various user and developer communities, and some external libraries are used. More details are available in the Acknowledgements section of the manual.
Q1.21 What project(s) support(s) TOPCAT?
TOPCAT has been supported by a number of grants and projects over its lifetime, nearly all ultimately funded by UK or European astronomy research organisations. In chronological order:
The work has mostly been done in the Astrophysics Group of the School of Physics at the University of Bristol in the UK (though some was done in various senses at the Institute of Astronomy in Cambridge and ARI in Heidelberg).
Q1.22 What are its future prospects?
TOPCAT has no assured long term funding, but has managed for the last decade or two to hop from one funding source to another without falling between the cracks. At time of writing things are looking positive for the near future: funding is secured until mid 2024 by grants from STFC, the UK's main funding body for astronomical research.
If because of terminated funding or for some other reason I cease work on it, another project or individual could take on some level of maintenance, support and/or development in my absence. I like to think that the code is sufficiently well organised and documented that this is a realistic possibility. Feel free to take a look to see whether you agree.
Q2.1 What do I need to run TOPCAT?
You need a Java SE (Java 2 Standard Edition) runtime environment version 8 or above (see Q2.2) and, usually, one of the TOPCAT jar files (topcat-lite.jar or topcat-full.jar, though it's available in other forms too). Since Java is available for almost all modern machines, TOPCAT will run on almost any computer with a screen, for instance Linux, Solaris, Mac OSX and MS Windows.
It will not run on a phone or iPad, or within a web browser.
Q2.2 What version of Java do I need?
Current versions of TOPCAT/STILTS require Java SE 8 or later. This Java version has been available since 2014, so it is likely to be installed already, and if not should be available, for any but the most ancient platforms.
If you need to update Java on OS X, for instance if your java installation
is Java 6, then apparently updating
the JDK rather than the JRE is a good idea, since otherwise the version
that runs when invoked using
/usr/bin/java is not updated.
Thanks to Jon Loveday for this information.
On other platforms, Oracle's JRE is recommended. If you do not already have this, you can obtain it from Oracle's Java SE page "Java Runtime Environment" is what you want.
Recent versions of OpenJDK have been reported to work tolerably well, though some people have also experienced problems with them. For the present I will continue to develop using Oracle's Java, so it remains the recommended platform.
If you are unable or unwilling to upgrade your Java installation, you can use an older TOPCAT version: v4.6-3 (2019) or earlier will run on Java 6, v4.3-2 (2015) or earlier will run on Java 5, and v3.5-2 (2010) or earlier will run on Java 4.
Q2.3 I get an error like "java.lang.UnsupportedClassVersionError"
This means that the Java version you have installed is too old to run TOPCAT. Since TOPCAT v4.7 (STILTS v3.2) you will need at least Java 8 ( Q2.1). Upgrade your java to a more recent version Q2.2 and try again.
Q2.4 What's the best way to download and run TOPCAT?
The recommended method is to grab
TOPCAT web page.
On a Un*x system you can also get (and chmod +x) the
shell script and put it in the same directory.
Then just invoke "
topcat" (if you have the shell script)"
java -jar topcat-full.jar" and you're off.
If you are tight for space, you can use
instead - it is missing only a few less important facilities.
If you have a Mac instead, you may prefer to get the topcat-full.dmg file instead, but note items Q4.4 and Q2.6 in this case.
If you are running an Ubuntu-derived Linux distribution, see the question about Debian Astro.
Q2.5 What about Debian Astro?
Since early 2017, the TOPCAT and STILTS packages have been included in the Debian Astro suite (thanks to the hard work of Ole Streicher). That means that in due course those packages will be made available in Debian-derived Linux distributions such as Ubuntu. In such distributions, you should be able to just do
# apt install topcat stiltsto get both packages. These should appear in Ubuntu 17.10 and Debian 10. This is not however guaranteed to give you the most recent versions.
Note however that these versions are not identical to the "official" releases available from http://www.starlink.ac.uk/topcat/. They should work OK, but the version may lag behind the official release, some of the external library versions are a bit different, a few of the more obscure features may be missing, etc. Version skew apart, differences in behaviour between the two are possible, but not expected to be significant.
Q2.6 How do I set command-line flags from Mac OS X?
If you have downloaded the
topcat.dmg file from the
web page and installed it on your Mac, you will be able to run it in a
Mac-like way by pointing and clicking. However, this does not permit
you to specify important command-line flags such as
-Xmx512M, not to mention specifying tables on the
command line to load at startup etc
(see the manual for
more information on command-line arguments).
To do that, you will have to run it from the command line.
Depending where you have installed the
topcat script should be present at a location like:
/Volumes/topcat/bin/topcatso invoking the script at that location or putting that directory on your path and invoking
topcatwith the relevant flags should do the trick.
Note: as at 2018
location doesn't seem to contain the
though I think it used to. This must be to do with MacOS application
installation, which I don't understand and can't easily experiment with.
If you do and you think you can help me fix it up, please get in touch...
Failing that, you should be able to find the
file somewhere. In that case, something like this ought to work:
java -Xmx512M -jar /Applications/TOPCAT.app/Contents/Resources/Java/topcat-full.jar
Another possibility if there are command-line flags that you want
to use on a regular basis is to edit the
file associated with the application bundle. You should find this
Apple provides documentation on the
Info.plist file format.
In particular if you want to
increase the heap memory
you can fill in the
<string> value of the
Java section with a value like "
Thanks to Frank Primini, Juan de Dios Santander Vela, Klaus Rübke and Nicholas Seymour for help with this question.
Q2.7 I have a Retina hi-res display on my Mac and TOPCAT looks blurry
Thanks to Christophe Deil for supplying this answer.
TOPCAT does work on Macbook Pro with Retina display, although some plots are drawn as bitmaps by the application and then scaled by 2x by the java implementation before being rendered to the Retina screen, resulting in unsharp image and text. Everything else (text, buttons, other plots) will look Retina-sharp if you start TOPCAT in high-resolution mode.
From the command line TOPCAT will start in high-resolution mode automatically. To make TOPCAT start in high-resolution mode when double-clicked on the icon in Applications or the Dock, you have to manually declare TOPCAT to be high-resolution capable. Add
<key>NSHighResolutionCapable</key> <true/>toward the end of the
/Applications/TOPCAT.app/Contents/Info.plistfile with the text editor of your choice, right before the closing
</dict></plist>lines (see also the previous question). Because Mac OS X caches the Info.plist content you have do a little trick to make it reload the Info.plist content change. Go to Applications in Finder and select TOPCAT. Type CMD+C, then CMD+V and you'll get a copy called "TOPCAT copy". Delete the original TOPCAT, then rename "TOPCAT copy" to TOPCAT. This seems fishy, but it is the standard suggestion for plist changes.
Finally you can check by right-clicking on TOPCAT and selecting "Get Info" in Applications, you should see the "Open in Low Resolution" checkbox disabled. A simpler check is to just start TOPCAT, if low-resolution mode is enabled you will see it right away because then everything in TOPCAT (text, buttons, plots) will look unsharp.
I may decide to change the default in the future, but for now there seems a question about whether setting hi-res might cause problems. Any feedback about positive or negative experiences with this is very welcome.
Q2.8 How do I set up TOPCAT to run on Microsoft Windows?
The following instructions have kindly been supplied by Clive Page for use with Windows XP:
C:\WINDOWS\System32\java.exeand they should be modified in the obvious way if this is not the case.
topcat-full.jarto a suitable location; the following instructions assume
topcat-full.jarand that you put it in your own "
My Documents" folder, but any folder will do.
topcat-full.jarin your own "
My Documents" folder:
C:\WINDOWS\System32\java.exe -jar "C:\Documents and Settings\your name\My Documents\topcat-full.jar"If you don't get this right first time it can be altered later, using the Properties window as described below. Note that you need quotation marks around the string after the
-jarswitch if it contains spaces, as it does in the example above. If you want to provide a switch for example to specify more memory, such as
-Xmx100Mfor 100 Megabytes, then this switch should be inserted before the
-jarswitch. Click NEXT, then you get prompted for a name: enter "Topcat" or whatever seems appropriate. Click FINISH and a new icon should appear on your desktop (with a Java logo).
java.exefile. You will probably find it convenient to alter this to some other folder, such as the one where your FITS files are generally located, or where your browser downloads them from the web. After editing it, finish by clicking Apply and OK.
Christopher Bishop adds some notes relating to Vista and Firefox downloads:
If you are downloading Topcat using Firefox on Vista it is important that you make sure that you set the Firefox download to the folder you want to save it in. If you don't then the topcat-full.jar will end up by default in a temporary file. If you then follow steps 3 and 4 above a Topcat icon will be created, but you may find Vista will have deleted your download in the temp file. At least it did in mine! The solution I found was to go to Firefox Tools - Options - Main and then look at the downloads window. In the downloads window there is a box that enables you to specify where you want the download to go. So I put
C:\user\yourname\documents(I would check in your documents folder that topcat-full.jar is really there and that Vista hasn't deleted it!)
Q2.9 Can TOPCAT be run as an applet?
No. It is possible that with some changes to the code it could be done, but I don't have any plans to do that. Applets are a bit out of fashion now, and in general TOPCAT seems a bit heavyweight to run inside a web browser (for instance it tends to need a lot of windows at once).
It can be run using Java WebStart, which is a technology that allows you to make one click on a web page and start the application (downloading it automatically first if necessary). You can see an example at topcat-lite.jnlp.
Q2.10 It doesn't seem to work!
Installing a java application such as TOPCAT is usually pretty straightforward, and there's not too much that can go wrong - at least compared to applications written in compiled languages such as C. If it's not starting, or failing to complete even very simple actions, the most likely explanation is that the version of Java you are using is broken or incomplete. You should try to install or reinstall Java for your system; see Q2.1 and Q2.2.
If you have a Mac, see also Q4.4.
If neither of those seems to be the problem, please mail me and ask for help.
Q3.1 What table format should I use?
The most efficient table format to use is FITS (either basic or "FITS-plus" - see Q3.2) specifically FITS BINTABLE extensions in uncompressed files on disk. If your tables are large, using FITS rather than some other format will both save memory (enabling you to work with larger datasets) and speed up table loading. For very large, and especially very wide tables (bigger than available RAM), the colfits format may be a better choice. If your data is initially in some other format (e.g. an ASCII-based one) it is often a good idea to convert once to a FITS-based format and subsequently use the FITS file within TOPCAT.
If your tables are small then the efficiency doesn't matter so much. VOTable and FITS-plus, and to a lesser extent basic FITS, can store rich metadata (units, UCDs, column descriptions etc) which may be something you want.
If you want to do processing of a table in some other language such as Python, IDL or Perl, a number of other formats are available. ECSV is good for interoperating with Python, Feather is efficient for interoperating with R, and CSV (Comma-Separated Values), while inefficient and having various problems, is generally a workable lowest-common-denominator solution.
Q3.2 What's FITS-plus?
TOPCAT can read and write two variants of FITS binary tables, "FITS-basic" and "FITS-plus" (these are just terms used by STIL/TOPCAT to distinguish the two variants). Both of these are perfectly legal FITS tables, but TOPCAT squirrels away some extra metadata in the primary HDU when using FITS-plus. In most cases, TOPCAT writes FITS-plus by default, and reads either. Nearly all of the time, you shouldn't need to worry about the difference.
There is more detailed information in the manual; see also SUN/252.
Q3.3 How do I read Multi-Extension FITS files?
If you load a FITS file, any table extensions it contains (BINTABLE or TABLE extensions) will be loaded as separate tables by default.
Q3.4 How do I read multi-table VOTable documents?
If you load a VOTable document, any tables it contains (<TABLE> elements) will be loaded as separate tables by default.
Q3.5 What size tables can TOPCAT cope with?
This depends a bit on one thing and another, but as a rule of thumb a few million rows and a few tens of columns should be quite usable while maintaining quite good performance. If you're having trouble with much smaller tables than this, there is almost certainly something you can do about it. For more details see Q3.6.
Larger tables are not out of the question; there are reports of interactive use with 300 million rows, and 20 thousand columns (not at the same time).
Q3.6 Help! I've run out of memory.
A lot of effort has gone into making TOPCAT scalable. Some data sets may simply be too large to process, but up to a few million rows it should be possible to use the program with good performance and without running out of resources. You may have to take some steps to ensure that this is so however. Some good tips are:
-diskflag. But since v3.5 TOPCAT uses an adaptive storage policy, and now this is rarely necessary.
Some operations which would run out of memory in TOPCAT can be done using STILTS, since it can stream data in some circumstances.
Q3.7 Can I save a Row Subset?
The Session save option in the Save Window can be used to save your entire session (for some or all of the loaded tables). Unlike the Current Table save option, this saves things like lists of subsets, hidden columns, etc, in addition to the usual table data and metadata. It can be used to save a single table or several in the same file. To reload the session data, simply reload the saved file into TOPCAT like a normal table.
An alternative way, for instance if you want to communicate the subset contents to a non-TOPCAT application, is to create a new boolean column representing the subset(s) to save, and save the table with that column. You could create the new column from the Subset Window using the Subsets|To Column menu item or equivalent toolbar button; this column then contained true/false according to whether each row is/is not in the subset in question. When the table was reloaded into TOPCAT, new subsets will be created for any such boolean columns.
Q3.8 TOPCAT won't read my VOTable
It is possible that there is a bug in TOPCAT, but it's
much more likely that there is an error in the VOTable.
The best thing to do in this case is to diagnose the problem by using
tool. Do not be afraid, this is very easy to do, something like:
% stilts votlint foo.xmlor
% topcat -stilts votlint foo.xmlThis will print WARNING or ERROR messages to the terminal describing questionable or illegal aspects of the file. You may be able to see what's wrong, and possibly fix it with careful use of a text editor from this information. Either way, if there are errors it would be a good idea to contact the originator of the VOTable with the
votlintoutput and tell them that they are generating broken VOTables.
Q3.9 Where has my VOTable metadata gone?
TOPCAT's data access is format-neutral - it has its own model of what constitutes table data and metadata rather than being tied to a particular data format. This is good in that it means it can work with tables in various different formats. However, it also means that data/metadata structure which is very specific to a particular data format may not be accommodated. Things like VOTable <GROUP> and <RESOURCE> elements fall into this category, except in some specific cases (e.g. where they related to TIMESYS/COOSYS elements).
To put it another way, both load and save operations in TOPCAT result in a sort of format translation (to and from TOPCAT's internal data model). Some things can get lost or changed in both of these stages, so unfortunately loading a table into TOPCAT and saving it without making any explicit changes can result in loss of data.
To put it a third way, TOPCAT is (amongst other things) a table editor, but if you think of it as a VOTable editor you may be disappointed.
Q3.10 Can I indicate missing upper/lower limits when plotting error bars?
There is no specific facility for displaying missing limits, but you can probably generate a plot that looks like you want it to. You can represent missing limits using the error bar style which resembles little arrow heads. Set up two subsets in the plot, one for points with error limits and one for points without. You can set up two data sets in the plot with the same point data but different error data, one containing the subset of points with limits and one containing the points without. For the points without limits, provide fake error values and use a suitable error plotting style, such as the error bars with the little arrow heads.
Q3.11 How can I join two tables with the same row numbers side by side?
TOPCAT's crossmatching facilities are fairly smart and flexible.
So much so that they can do stupid, easy things as well as clever, hard ones.
If you have tables that want joining together without reordering the
rows, choose the Exact Value algorithm and use the
magic value "
$0" for the Matched Value Column
for each input table.
$0 evaluates to the row index in
each case, so that all the row 1's will be joined together, as will all
the row 2's etc.
In STILTS it's easier, there is a command that does just this:
Q3.12 The crossmatching is giving the wrong results!
The crossmatching code in TOPCAT/STILTS is quite well tested, and reports of actual bugs are rare. However, I take the correctness seriously, and if you think you've found an anomaly, please report it.
The following bugs affecting correctness of crossmatching have been reported in recent years:
<n>d_err/ <n>-d Cartesian with Errors
skyerr/ Sky with Errors
2d_ellipse/ 2-d Cartesian Ellipses
skyellipse/ Sky Ellipse
healpixRingIndexfunction, which resulted from a bug in the
vect2pix_ringfunction of the underlying PixTools HEALPix library. This function is used during sky crossmatching, so seems possible that it could have resulted in some missed associations. If so, the problems would have been for associations with a point on each side of the longitude=0 line in the equatorial half of the celestial sphere. Tests have failed to show up any actual match failures, so it looks like this did not affect match results, but it's hard to say for sure that it could not have happened. This bug is fixed in any case at TOPCAT v4.5 and STILTS v3.1-1.
No other crossmatching correctness bug has been reported in recent years.
One thing which can be confusing is matching objects in crowded regions, since in crowded regions (where the typical distance between positions is similar to or less than the match radius) the results depend strongly on the details of the matching criteria.
Note in particular that TOPCAT's "Best Match Symmetric" mode
is symmetric between tables.
If you select "Best Match Symmetric" for the Match Selection toggle
(or equivalently "
find=best" in STILTS
tasks), then each row from either table will only appear a maximum
of once in the output table. That means if object A in table 1 is a
match for object B in table 2, it cannot be a match for any other objects
in table 2. This may lead to fewer matches than you were expecting.
If what you want to do is to find, for each object in table 1, the
closest match in table 2, you should use one of the asymmetric options:
"Best match for each Table 1 row" (or "Table 2 row") in TOPCAT's
Match Selection chooser, or "
find=best1 (or "
Note that these options were only introduced in TOPCAT v3.9/STILTS v2.4,
prior to that only symmetric best and all options were available.
So, if you have an earlier version and want to do this, you should upgrade.
If you've got doubts about the correctness of a crossmatch, it's a very good idea to plot the results to see whether it's doing what you think or hope it is. Since v4.1 this is usually fairly easy; see the manual.
Q3.13 Can I crossmatch more than 2 tables at once?
Yes. The multi-table crossmatch options are available from the Joins menu in the control menu. They don't appear on the toolbar just because they would take up too much space. Currently a 4-way match is the most you can do at once. But that's only because nobody has asked me for more. If you want the option to match more than that, let me know and I will add higher-numbered options.
Somewhat more flexible multi-table crossmatches are available
using the STILTS
Equivalent functionality will be added to TOPCAT at some point.
Q3.14 Can I concatenate more than 2 tables at once?
No. You can do this using the STILTS
Q3.15 Can TOPCAT read SExtractor output?
It cannot read the ASCII files that SExtractor outputs by default. If you want to use TOPCAT with the output from SExtractor, set it to write its output as FITS tables. You can do this by setting SExtractor's CATALOG_TYPE parameter to "FITS_1.0".
Q3.16 Can I script TOPCAT?
No. The only ways you can control TOPCAT itself are
However TOPCAT's sister application STILTS is a set of command-line tools which provide much of the same functionality as TOPCAT. You can write scripts using these applications if you want to automate the kind of table processing that TOPCAT can do. For generating STILTS plots corresponding to TOPCAT plots, the Stilts Control in TOPCAT's plot windows may be useful.
Q3.17 Can I save/restore the state of a TOPCAT session?
Since version 3.6 (August 2010), yes you can. Use the Session option in the Save Window to save, and load the table that outputs in the normal way to restore.
Note however that this saves (most of) per-table state, and not everything, so for instance currently displayed plots will be lost. This may be improved in a future release.
Q3.18 Can I see a log of the actions TOPCAT has performed during a session?
No, I'm afraid not.
Q3.19 Can TOPCAT talk to other tools/processes?
Yes, by using SAMP or PLASTIC.
Q3.20 What's SAMP and/or PLASTIC all about then?
SAMP is the Simple Application Messaging Protocol.
It allows different tools, usually running on the same desktop, to talk to each other, for instance by sending tables from one to another. This topic is discussed in more detail in the manual.
Briefly, if a SAMP "hub" is running then some of TOPCAT's controls allow you to send instructions or data to other tools. Usually these options are controlled from an Interop menu on the relevant window.
A SAMP hub may already be running as part of another application (such as Aladin). If it is not, TOPCAT usually starts one internally when it starts up.
PLASTIC is the PLatform for AStronomical Tool InterConnection; it was an earlier protocol that did effectively the same thing as SAMP, but is now more or less obsolete.
Q3.21 Can I calculate the difference between column values in adjacent rows?
The expression language used by TOPCAT works entirely within a single row, so doing things like adding a new column B whose value in row n is the difference between the values of another column A in row n and its value in row n+1 is not directly possible.
There is a hack which you can use to achieve this however.
If you join (match) the table to itself so that the same column appears
twice in the resulting table, but offset by one row, you can then
do calculations involving both values. The join can be done
in the Pair Match Window,
by using the Exact Value match algorithm,
using the same table for both inputs, with the
Matched Value Column
set to "
$index" and "
$index" (or equivalently "
$0") token is
a magic column which evaluates to the row number in the table.
You can then add a column B with an expression like "
to get the difference. If required, you could then match the part of the
table containing the new column back to the original table.
Yes this is a bit of an effort. Here is a STILTS command which effectively does what is described above:
stilts tmatch2 in1=table.fits in2=table.fits \ matcher=exact values1='$index+1' values2='$index' join=1or2 \ ocmd='addcol b a_2-a_1'
Q4.1 I can't see my Documents/Downloads/etc folders on MacOS Catalina+
It seems that in Catalina (and later versions?) of MacOS, certain directories are effectively invisible from the various file choosers, e.g. the Filestore Browser used in the Load and Save windows.
This seems to be related to new security restrictions introduced in this version of MacOS; I think it might be to do with the way that TOPCAT is launched from the .dmg file (further reading for enthusiasts here and here). I currently can't work out how to solve it, but some workarounds are possible: from the Load and Save windows, there is the option to use the System Browser as an alternative to the Filestore Browser. If you do that, you should get the standard MacOS Chooser, which ought to be able to see all the directories. Alternatively, if you start topcat using
java -jar topcat-full.jarrather than by clicking on the icon, it seems to work OK.
Something else that might work is accessing System Preferences/Security & Privacy/Privacy and giving topcat Full Disk Access.
Apparently, if you use the System Browser to look at these folders once, then subsequent access with the Filestore Browser works as well, maybe even in different TOPCAT invocations; see this report.
Thanks to Davide Lena (SRON) Marina Kounkel (U.Michigan), and Jarle Brinchmann (Leiden) for help investigating this.
Q4.2 Overwriting mapped FITS file causes Java crash
If a FITS file has been mapped for reading (this is the normal way that uncompressed FITS files on disk are read) and it is overwritten, either by TOPCAT or by some other program, it can result in a crash of the Java Virtual Machine (ugly error message to the console and instant death of TOPCAT). TOPCAT will not normally perform such an overwrite, but it can't stop you doing it from outside the program.
The best thing is: don't overwrite FITS files currently in use.
However, you can work around this by deleting the file before you overwrite it,
or by loading it into TOPCAT using a
file:-type URL, e.g
file://localhost/foo/bar/cat.fits" rather than
/foo/bar/cat.fits". This will load without mapping
(and as a result will be much slower for large files).
Q4.3 Expressions containing self-references cause trouble
If an algebraic expression for a synthetic column or row subset
contains a reference to itself via a
the program may hang. It might recover - if you can edit the expression
so it doesn't contain such a reference any more, do so.
In version 4.6-3 and later, TOPCAT will usually prevent you from doing
Q4.4 On MacOS the .dmg file is reported "damaged" or "can't be opened"
On the Mountain Lion release (and later?) of Mac OS X, attempting to install the topcat-full.dmg file apparently can result in the message:
"TOPCAT is damaged and can't be opened, you should eject the disk image"or maybe
"TOPCAT.app can't be opened because Apple can't check it for malicious softwareor
"TOPCAT can't be opened because it is from an unidentified developer"
It seems that in Mountain Lion and later, security settings (a.k.a. GateKeeper) default to not letting you run anything from anywhere other than the Mac App Store or from signed applications. Unfortunately, this results in it reporting java-based non-signed applications as "damaged".
There are three? workarounds for this (though that may be MacOS-version dependent). Apparently, opening the file by using Ctrl-click rather than double-click will allow you to add a security exception to trust this file. If you do this once, subsequent usage should work in the usual way.
Another option is to eliminate the quarantine flag for the file from the command line. If TOPCAT is in the current path, you can type
xattr -d com.apple.quarantine TOPCAT.app
Alternatively, you can (perhaps temporarily) change one of the security settings by going to the Apple menu | "System Preferences..." | "Personal" section | "Security & Privacy" | "General" pane' and changing the setting Allow applications downloaded from: to Anywhere. You can run TOPCAT, and a dialog will be shown telling you the application is not signed. Once you accept it, you can revert, if you want, your security settings to the default setting, Allow applications downloaded from: Mac App Store and identified developers.
I think the underlying problem is that I'm not a trusted Apple developer, because I don't want to pay the non-negligable amount of money that costs. If anybody knows otherwise how to fix it, advice is welcome.
Thanks to various people who have helped with answering this question, including Juan de Dios Santander Vela, Jarle Brinchmann and Marina Kounkel.
Q4.5 Tables are scrambled when scrolling
In some cases apparently the table viewer window and other JTables displayed can get their pixels scrambled when they are scrolled. This must be some kind of bug in Java or maybe some display drivers. It seems to have more than one possible cause.
If the scrambling happens when you scroll to the right, the issue may be the one described here; to fix it try running with one or other of these system properties set:
Q4.6 The text on my high-resolution screen is too small
Some users report that running TOPCAT on high-resolution (4k, HiDPI, ??) screens results in a GUI that comes out too small, e.g. including fonts that are hard to read. The details of this problem seem to depend on exactly what hardware and software you have. Here are some measures that might fix it:
GDK_SCALEenvironment variable, which is set to an integer scaling factor. (Does this only work in Gnome? If so, does setting
GDK_SCALEby hand in non-Gnome environments help?)
java -Dsun.java2d.uiScale=2 -jar topcat-full.jar(see system properties). In principle this might accept fractional values such as 1.5, but at least in some (most?) cases it seems to be restricted to integer values.
GDK_DPI_SCALEbe set in some circumstances as well?? )
If you have any experience on what does and doesn't work in different environments, please get in touch so I can improve the advice here.
Q5.1 I can't find a resource in the NVO/VAO registry from the Cone/SIA/SSA/TAP window
Between mid-2010 and mid-2014 searching the NVO (a.k.a. VAO) registry at STScI did not work. This turned out to be the result of bugs in both TOPCAT and the STScI registry, as finally uncovered by the estimable Menelaus Perdikeas. In version 4.1 and later, these searches should work. In version 4.2 and later, the RegTAP protocol is used by default, and the problem ought not to arise.
Q5.2 It crashes immediately on startup, or when I open the Load window
There is a bug which may be seen by people who are set up to use Starlink applications as well as TOPCAT. It may provoke messages like:
no such constant AST__KYCIRor
# # An unexpected error has been detected by HotSpot Virtual Machine: # # SIGSEGV (0xb) at pc=0xf79af28b, pid=5153, tid=3888896928
You should be able to fix it by unsetting the LD_LIBRARY_PATH environment variable, for instance if you use a C-like shell:
% unsetenv LD_LIBRARY_PATHor for bash/sh:
$ unset LD_LIBRARY_PATH $ export LD_LIBRARY_PATHIf you continue to have problems, please contact me and I'll try to provide more help.
Q5.3 3D error bars extending outside the box are invisible
In the Cartesian/Spherical 3D plots, if an error bar would extend outside of the box/sphere then it is simply not plotted, so that even the part which is within the box/sphere is not visible.
Q5.4 Scisoft 7.0 invocation script broken
TOPCAT is included in the ESO
Scisoft software bundle.
topcat" invocation script included with that bundle
does not have the same functionality as the one documented in
so for instance command-line
arguments cannot be supplied to it.
If it's not doing what you want, you can run using
java -jar topcat-full.jar.
This problem should be fixed in Scisoft releases after 7.0.
Q5.5 I get an error about a SizingScrollPane on my Java 1.7 Mac
On some Macs running java 1.7 (Mountain Lion??), doing things like trying to open the table or statistics windows can give you a stack trace like:
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException at uk.ac.starlink.topcat.SizingScrollPane.getConfig(SizingScrollPane.java:81) at uk.ac.starlink.topcat.SizingScrollPane.getPreferredSize(SizingScrollPane.java:40) ...I can't tell if this is my fault or OSX's, but it is fixed in version 4.0 and later.
Q5.6 My plot axis labels come out backwards on OS X!
It seems that in recent versions (java 1.7?) of MacOS X,
the text written sideways up the Y axis of plots sometimes
comes out written backwards, e.g. it appears to read
EDUTINGAM_B" rather than "
In fact any text written non-horizontally is placed wrong
(e.g. sky axis labels).
I am pretty sure this is a Mac Java bug, which I hope will fix itself
at some point (does anybody know how to report this kind of thing?).
The new-style plotting windows in v4.1 can work around this issue (I believe) which seems to go away if you use antialiased text. As well as getting the text the right way round it does look a bit nicer like this, but it is also perceptibly slower. TOPCAT attempts to determine if its running in MacOS, and if so it sets antialiasing on by default. However at version 4.1, it doesn't always guess right. You can turn antialiasing on/off by using the Antialias checkbox in the Grid tab of the Axes control in the various plot windows, so try that if your labels are backwards.
In version 4.2 and later, text antialiasing will be set using the Antialias text type in the Font selector instead. By then, it ought to be better at guessing OS, so you hopefully will not need to worry about it.
Q5.7 When I load an LDAC FITS file I can't see my data
Files in the LDAC FITS format are organised in an unusual way with the first (LDAC_IMHEAD) table extension containing a single-cell table where the cell's content is an array of FITS headers. This confuses TOPCAT into thinking that the file is in colfits format, which in turn means it ignores the second (LDAC_OBJECTS) table extension, and that's the one you probably want to look at.
A hack is present in TOPCAT v4.2-2 and later to avoid this,
but there are various ways to work around it anyway.
One is to select "
FITS" (rather than "
as the input file format.
Another is to use the
to examine the file and select the table you want.
You can also select the correct table by name or HDU index as, e.g.,
D1_r_sex.cat#2", either on the command line or
using the Position in file field at the bottom right of the
Q6.1 I still don't know...
Try consulting the manual. Or at least the relevant part - click the Help button in the window you're in for some context-specific documentation. The Find field in the Help Window is also good for locating the right part of the documentation.
Q6.2 It's not in/I'm too lazy to read the manual.
The best thing is to subscribe to, then send a message to, the topcat-user mailing list (if it's a question you think others might be interested in too) or to email@example.com (if it's quite specific). I am usually monitoring the mailing list, and you will probably get a reply quite soon.