How to create an exporter
In this example we show how you can implement you own exporter and add it to DHIS 2.
Background
DHIS 2 abstracts away how to export with the Exporter interface.
public interface Exporter
{
/**
* Export the objects retrieved by the given Specifications to the given
* OutputStream
*
* @param specifications Collection of Specifications representing the objects to be exported
* @param stream OutputStream to export to
* @throws ImportExportException
*/
void export( Collection<Specification> specifications, OutputStream stream )
throws ImportExportException;
}
As explained in the code, the export method takes two arguments, a collection of Specification objects an a stream to write exported data to.
The stream is an attempt at abstracting where export should be done to. The stream can point to file, a network stream for download, into another application, into a database or other possibilities. By using a stream, the actual export implementation doesn't need to know where the exported data are going.
The Specification object is a tad more complicated. It's an attempt to abstract away an SQL query with a series of WHERE clauses. The object defines the class of the objects the Specification is supposed to apply to, for example DataValue.class or OrganisationUnit.class. In addition to the class, it defines a set of constraints on that kind of object, effectively a set of AND statements in a where clause.
For example, one could envision the following SQL statement:
SELECT * FROM organisationunit
WHERE openingDate = '2006-03-01'
AND parent in (id1,id2,id3);
and construct a Specification in this way:
Specification spec = new Specification( OrganisationUnit.class );
spec.addConstraint( "openingDate", new Date(106,3,1) );
Collection parents = new ArrayList ();
parents.add( ou1 ); parents.add( ou2 );
parents.add( ou3 );
spec.addConstraint( "parent", parents );
The addConstraint method takes the name of a field in the object to be retrieved and then either a single object value or a collection of values.
The Specification objects are designed to be used with the dhis-service-dataprovider module:
Iterator it = dataProvider.getValueIterator(spec);
In this example, the dataprovider will construct a Hibernate query for OrganisationUnit objects (the organisationunit table), add the constraints to the query and execute it. The iterator will contain all OrganisationUnit objects that conform to the specification, i.e. that both have the name "foo" and either ou1, ou2 or ou3 as parent.
Note that if you create a Specification on for example OrganisationUnit.class and do not specify any constraints, the dataprovider will give you all OrganisationUnit objects in the database.
Creating the exporter
Creating the exporter is simple. All you need to do is to implement the Exporter interface located in the module dhis-api, and the org.hisp.dhis.importexport package.
public class CSVOrganisationUnitExporter
implements Exporter
{
public void export( Collection<Specification> specs, OutputStream stream )
throws ImportExportException
{
}
}
A common case is to add a dependency to the dataprovider and use it to loop through the values it gets:
public class CSVOrganisationUnitExporter
implements Exporter
{
private DataProvider dataProvider;
public void setDataProvider( DataProvider dataProvider )
{
this.dataProvider = dataProvider;
}
public void export( Collection<Specification> specs, OutputStream stream )
throws ImportExportException
{
PrintWriter writer;
try
{
writer = new PrintWriter( new OutputStreamWriter( stream, "UTF-8" ) );
}
catch ( UnsupportedEncodingException e )
{
throw new ImportExportException( "Failed to create UTF-8 writer", e );
}
Iterator valueIter;
OrganisationUnit unit;
for ( Specification spec : specs )
{
valueIter = dataProvider.getValueIterator()
while ( valueIter.hasNext() )
{
unit = (OrganisationUnit) valueIter.next();
}
}
}
}
In the example above, we also demonstrate how you can wrap the OutputStream in different other types of Streams or Writers. What the Exporter does to the Stream, i.e. how it actual writes data to the stream, is hidden from the class calling the Exporter.
When exporting, you can basically format the object to be exported anyway you wish. Examples include using XStream to write the object as XML or pulling out it's instance variables as CSV. In the example below, we'll do CSV.
writer.writeln(
unit.getName() + ','
unit.getOrganisationUnitCode()
);
Defining the exporter bean
In order to use your exporter, you'll need a bean for it. The bean declaration can look like this:
<bean id="org.hisp.dhis.importexport.csv.CSVOrganisationUnitExporter"
class="org.hisp.dhis.importexport.csv.CSVOrganisationUnitExporter">
<property name="dataProvider"
ref="org.hisp.dhis.dataprovider.DataProvider"/>
</bean>
Note that your exporter can have any number of properties which you can specify here, for various help classes, formatters, converters, etc. A concrete example is an XML exporter using XStream and a series of Converter objects for the objects you're exporting.
Defining an exporter plugin
ExporterPlugins are the objects that are shown in the drop down boxes in the web GUI. Each plugin defines a name and a reference to the exporter it's associated with.
<bean id="orgunitCsvExporterPlugin"
class="org.hisp.dhis.importexport.ExporterPlugin">
<property name="name" value="organisationunit_csv_exporter"/>
<property name="exporter"
ref="org.hisp.dhis.importexport.csv.CSVOrganisationUnitExporter"/>
</bean>
Adding the ExporterPlugin
ExporterPlugins have to be registered with the ExporterPluginManager in order to become visible in the GUI. GUI Actions will typically ask the ExporterPluginManager for a list of available plugins.
There are two primary ways of registering your ExporterPlugin. One is to add it directly to the ExporterPluginManager defined in the beans.xml file in dhis-service-importexport (src/main/resources/META-INF/dhis/). Just add a reference to your ExporterPlugin at the bottom of the list:
...
<bean id="org.hisp.dhis.importexport.ExporterPluginManager"
class="org.hisp.dhis.importexport.DefaultExporterPluginManager">
<property name="exporterPlugins">
<list>
...
<ref bean="orgunitCsvExporterPlugin" />
</list>
</property>
</bean>
...
The other is to use a MethodInvokingFactoryBean, and add your ExporterPlugin to the manager above dynamically at runtime. If you define your own module, you can add the following to it's beans.xml file.
<bean class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject">
<ref bean="org.hisp.dhis.importexport.ExporterPluginManager"/>
</property>
<property name="targetMethod">
<value>addExporterPlugin</value>
</property>
<property name="arguments">
<list>
<ref bean="orgunitCsvExporterPlugin"/>
</list>
</property>
</bean>
The advantage of the second approach is that dhis-service-importexport would not need to have a direct dependency to your module to be able to use your exporter. Note however, that you can't run dhis-web-importexport standalone and still see your plugin using this method.
Decorating the exporter
It is possible to create exporters that simply decorate another exporter, adding some sort of functionality or preprocessing data. Right now DHIS 2 has one such exporter called ZipExporter. This exporter will wrap the OutputStream in a ZipOutputStream which means that the export will result in a zip file. The following is an example of how to enable this.
<bean id="orgunitCsvExporterPlugin"
class="org.hisp.dhis.importexport.ExporterPlugin">
<property name="name" value="organisationunit_csv_exporter"/>
<property name="exporter">
<bean class="org.hisp.dhis.importexport.ZipExporter">
<property name="decoratedExporter"
ref="org.hisp.dhis.importexport.csv.CSVOrganisationUnitExporter"/>
<property name="entryName" value="organisationunits.csv"/>
</bean>
</property>
</bean>
In this example the exporter plugin doesn't reference our exporter directly. Instead it references an instance of the ZipExporter which again wraps our exporter in the decoratedExporter property.
The entryName property refers to what the name of the file inside the zipped file will be. In this case, when you unzip the resulting zip file, you will get a file called organisationunits.csv.
Using a formatter
In the case of CSV and other text formatting, you might want to separate the export process from the actual formatting of the individual value to be exported. There is a Formatter interface which defines this capability.
public interface Formatter
{
/**
* Format the given object as a String
* @param o Object to format
* @return String representation of the object
*/
String format( Object o )
throws ImportExportException;
/**
* Format the given object as a String, using aliases for certain fields.
* @param o Object for format.
* @param aliases Map of aliases mapping fields in the object to other objects
* @return String representation of the object
*/
String format( Object o, Map<String,Object> aliases )
throws ImportExportException;
}
The interface supplies two methods, the simple format method which takes the object to format as an argument and returns a String with the formatted results. The second method also allows an optional map of aliases. For example, the DataValue class has a reference to a Source, but in many cases this Source will be a stand in for an OrganisationUnit. In such a case, you can do the following:
OrganisationUnit unit = organisationUnitService.getOrganisationUnit(dataValue.getSource());
Map<String,Object> aliases = new HashMap<String,Object> ();
aliases.put( "source", unit );
String formattedValue = formatter.format( dataValue, aliases );
This is a way of telling the formatter that if it wants to format the "source" property of the DataValue object, it should refer to the object in the aliases map instead. This enables the formatter to move down the object graph in the unit object if it wants to.
Using CSVFormatter
Basic usage
The CSVFormatter class is a generic class for formatting objects as CSV. When you wish to format an object, configure the formatter with a pattern and send it the object to format. The CSVFormatter will parse the pattern and use it to determine which properties of the object to put into the resulting CSV string. The idea of the Formatter is that you can devise any number of CSV formats you want to export, but you'd still only need this class. The pattern changes, but the class stays the same.
As an example, consider the OrganisationUnitExporter we defined above. It formatted an OrganisationUnit as CSV by extracting it's name and organisationUnitCode. If you wanted to use the CSVFormatter class to do this, it would look like this:
CSVFormatter formatter = new CSVFormatter ();
formatter.setPattern( "name,organisationUnitode" );
String formattedValue = formatter.format(unit);
Each component in the pattern must be the name of a field in the object. It is possible to dot your way through other objects the first object is referencing, for example:
Which would mean the name field of the object found in the parent field of the object being formatted.
Assuming the unit object was made like this:
unit = new OrganisationUnit ( ... "OrgUnitName", ... "OrgUnitCode" ...);
The resulting formattedValue would look like this:
As the formatter object specifies, this class also supports aliases.
You can choose to change the delimiter by calling the setDelimiter(delimiterString) method on the Formatter. The default delimiter is a comma (',').
So, how does it work? The CSVFormatter uses OGNL to parse the expressions and retrieve values according to the expressions. Like in Velocity, the root of the expression is implied as the object being formatted. I.e., this expression in the pattern:
Means:
Note that because of the OGNL usage and the expression parsing, this class will be slower than counterparts using hard coded accessing of fields in the object.
Using graph expressions
It is possible to define expressions that traverse down an object graph, getting data for each object it encounters in the graph. The format of such an expression is:
graph(pathExpression):(selectionExpressions)
For example, if you wanted to move down the parent graph of an OrganisationUnit object, pulling out the name for each object in the graph, it would look like this:
Assuming that the object has two ancestors, OrgUnit 1 and Org Unit 2, the resulting string would be:
Note that the values are "reversed", i.e. processed in tree order from the root and down to the object being formatted.
This functionality is particularly useful if you want to format a hierarchy of objects, for example for pivot tables.
Configuring the CSVFormatter as a bean
The CSVFormatter is designed to be used as a bean. This way you can put your pattern in your configuration file and change it without recompiling the system. If we wanted to enable the CSVFormatter for the OrganisationUnitExporter, we could write the following:
<bean id="orgUnitCsvFormatter"
class="">
<property name="delimiter"><value>,</value></property> <!-- Optional, ',' is default -->
<property name="pattern">
<value>name,organisationUnitCode</value>
</property>
</bean>
<bean id="org.hisp.dhis.importexport.csv.CSVOrganisationUnitExporter"
class="org.hisp.dhis.importexport.csv.CSVOrganisationUnitExporter">
<property name="dataProvider"
ref="org.hisp.dhis.dataprovider.DataProvider"/>
<property name="formatter"
ref="orgUnitCsvFormatter"/>
</bean>
You'd change your OrganisationUnitExporter class in the following way:
public class CSVOrganisationUnitExporter
implements Exporter
{
...
private Formatter formatter;
public setFormatter(Formatter formatter)
{
this.formatter = formatter;
}
...
public void export( Collection<Specification> specs, OutputStream stream )
throws ImportExportException
{
... in the while loop:
writer.writeln(formatter.format(unit));
...
}
...
}
