ESA-VO DALToolKit (Images, Spectra)
Aurelien Stebe & Inaki Ortiz
SIAP (Simple Image Access Protocol) and SSAP (Simple Spectrum Access Protocol) are two successful protocols in the Virtual Observatory context. Due to the simplicity and flexibility of these two simple protocols, it is easy to adapt existing archives to be VO compliant. It is also easy to make use of the available protocols to consume images and spectra in the VO context with appropriate clients and tools.
The goal of this tutorial session is to ingest and publish spectra / images according to the SSAP / SIAP standards. Participants will be able to locally create their own database, ingest test FITS files and publish them making use of the software and the support we will provide. Later on, people who wish to work with their own data will be able to perform the necessary tasks to achieve such result.
During this tutorial session, the user will follow all the necessary steps to create real SSAP / SIAP services on their machine. We will provide a few test FITS files which have to be ingested in a database according to the rules covered by a configuration file. Users will also be able to configure and install their own service which will display the data in SSAP /SIAP formats.
- Phase 1
- Check your configuration and get all the necessary software
- DALToolKit configuration files description and editing
- Ingestion, compilation, deployment and testing of the sample
- Phase 2
- Same process is repeated using participants FITS files
- Deployment on Workshop server and registration of service
- Phase 3
- Further the mapping efforts with participants custom data
- Architecture, source code, extension points, questions, ...
Check your configuration
First of all, you should check that you have installed correctly all the necessary software
common to all the tutorial sessions. Check particularly that you get the right outputs to the verification steps at the bottom of the page. Ask a tutor for help if you get any problem.
Get the necessary packages
Then, you should retrieve on your machine, in a working directory all the DALToolKit software packages
. These include the DALIngestor, used as the name suggest during the ingestion process, the actual DALToolKit server source package,
, the test FITS data you will try to use first.
Note that two flavors of the DALIngestor and DALToolKit exist : one for MySQL users (
) and on for PostGreSQL users (
). The DALIngestor is quite specific to the RDBMS you are using and hence required two different versions. The DALToolKit source code is actually identical in the two flavors, be we prepared them with the correct specific database configuration for ease of use. Note that the PostGreSQL flavors are only available on the online DVD
Deploy the DALIngestor and data
Now we will deploy the DALIngestor under the Tomcat application server and make the FITS data files available online.
You should copy the DALIngestor (
file) inside the Tomcat
Copy also the FITS data file directory under the same
directory : it is now time to choose if you will start by creating a SIAP service (
directory) or a SSAP service (
You may now start your Tomcat server if it is not running already (go to http://localhost:8080/
to check if it is) : type
in a terminal
Create a new database in your RDBMS
Start your database server if it is not running already and create a new database for your DALToolKit service. Use the administration GUI or login directly into your database server using the Shell application. Under the shell terminal the command to create a new database is :
CREATE DATABASE database_name;
Create a new database called
depending on your service type.
Edit the DALToolKit configuration files
We will now configure the DALToolKit service according to the test data you are using the type of service you wish to build.
Go to the DALToolKit directory inside the
directory. If you are using the Images use the
directory, if you are using the Spectra use the
file and check / edit the different values according to your database installation and set-up.
files and check / edit the various configured
Here is the list of possible configuration values :
- ColumnName indicates the source of the data and can take the following formats : 'COLUMN_NAME' or 'SQL_FORMULA AS NAME'
- ParamName controls the input capability (delete to ignore)
- UTypeName controls the output capability (delete to ignore)
- FitsKeyword is not required/used by the DALToolKit server, but is by the DALIngestor
- Description is used in the output of 'FORMAT=METADATA' calls
- DataType, ArraySize, Unit and UCD should follow the VOTable v1.1 Recommendation, they are used for the VOTable output
- Values is used in the output of 'FORMAT=METADATA' calls, and can take the following formats : '[min,max]' or 'val1,val2'
Compile and deploy the DALToolKit server
To deploy the DALToolKit Server you have to compile it first. To compile the source code (Java) you will be using Ant, a Java based build tool. Once you have compiled the DALToolKit Server source code, you have to package it as a standard web application file (WAR) and install it into the web application server (Tomcat) were the service will be finally up and running.
Go back to the root of the DALToolKit directory and run the following ant commands :
ant build to check everything is compiling correctly
ant deploy to pack the service and send it to Tomcat
- note that you may also use
ant undeploy or
ant redeploy as needed
The Ant script will ask you for various inputs like the name of the configuration directory you wish to use (
), where to deploy (
) and your Tomcat password.
You may now open http://localhost:8080/ULSSIA/
to check the service is there.
You cannot query the service yet as we still need to ingest the metadata and create the database table.
Use the DALIngestor to extract the metadata
Open the DALIngestor application at : http://localhost:8080/DALIngestor/
or http://localhost:8080/D DALIngestor4PSQL/
Supply it with all the necessary information (service type, FITS files locations, configuration files) and launch the ingestion process.
Test your final DALToolKit service
You may now re-open your DALToolKit service and submit a test query to check everything worked fine.