CUD REST interface - Client user guide

Parameters are included in the query string.  These should be URL-encoded (there are plenty of free URL-encoding utilities available both on and off the web).

Allowed parameters are as follows:

q

This is the parameter describing the Solr query, and is the only mandatory parameter.

Legal values are empty (no data will be returned), or a legal Solr query.

An example of the Solr query cud\:cas\:sso_username:unit1234, when encoded:

q=cud%5C:cas%5C:sso_username:unit1234

The Solr query cud\:cas\:sso_username:unit1234 AND cud\:cas\:current_affiliation:oucs, when encoded:

q=cud%5C:cas%5C:sso_username:unit1234%20AND%20cud%5C:cas%5C:current_affiliation:oucs

fields

The fields parameter allows the requestor to reduce the set of values returned for each record to only those attribute fields that are specified.

Legal values are a comma-separated list of attribute names.

An example of the fields parameter used to specify the Oxford email and CUD ID (cud:cas:oxford_email,cud:cas:cudid):

q=cud%5C:cas%5C:sso_username:oucs0098&fields=cud:cas:oxford_email%2Ccud:cas:cudid

format

The format parameter in CUD queries is optional, and specifies the format in which the requested data will be returned. It may be omitted, in which case the default 'csv' format will be used.

Legal values are 'csv' (the default), 'xml', 'json', or 'text'. If 'text' is chosen you can optionally set the 'delimiter' and 'qualifier' parameters to further customise the output.

An example of the format parameter being used to specify data returned as XML:

q=cud%5C:cas%5C:sso_username:unit1234&format=xml

delimiter

The delimiter parameter in CUD is optional, and only relevant when the 'text' format is used. For other formats this parameter will be ignored. The first character of the value set for this parameter will be used as the field delimiter. If the text format is set, and this parameter is omitted, a comma will be used as the delimiter.

An example of the text format using a specific delimiter (|):

q=cud%5C:cas%5C:sso_username:unit1234&format=text&delimiter=%7C

qualifier

The qualifier parameter in CUD is optional, and only relevant when the 'text' format is used. For other formats this parameter will be ignored. The first character of the value set for this parameter will be used. The purpose of this parameter is generally to escape the field. If the text format is set, and this parameter is omitted, no additional qualifier will be used.

An example of the text format using a specific delimiter (|) and qualifier (@):

q=cud%5C:cas%5C:sso_username:unit1234&format=text&delimiter=%7C&qualifier=@

history

The history parameter in CUD queries is optional, and indicates whether you wish the response to contain historical values for attributes and affiliations. You should only request this if it is required, and the default is not to include history. Include history only if you are requesting data in XML or JSON format and want it to be richly structured (see the CUD UI User Guide for further details).

Legal values are 'y', 'n' or omitted entirely.

An example of the history parameter used to request historical data in an XML format:

q=cud%5C:cas%5C:sso_username:unit1234&format=xml&history=y

encoding

Normally the Character Set encoding of the response returned by the query defaults to the modern standard set referred to as UTF8.  You should not usually need to change this.  However, if your application calling the query does not support UTF8, you can specify an alternative with the encoding parameter.  The supported Character Sets are those available from the Java SE 8 platform used by the CUD web-service, the Java SE documentation provides a comprehensive list.  The encoding parameter specifies the canonical Character Set name from that list.
 
A typical case where this is needed is if the calling program supports the ASCII (US-ASCII) Character Set, which is only a subset of UTF8. An example of this usage would be:

q=cud%5C:cas%5C:sso_username:unit1234&format=xml&encoding=ASCII

This will request the server to modify the response output to ensure that the response characters all conform to the specified encoding. In this case for ASCII, all of the output characters which are actually UTF8 but not ASCII will be changed to the ASCII question mark character "?".

This also handles the situation where UTF8 characters which are actually made up of multiple underlying bytes, for example the character "ö" is a 2-byte UTF8 character (0xC3B6). The encoding=ASCII parameter will ensure that two bytes are replaced by a single "?".

For format=xml, the CUD web-service will also modify the XML header tag accordingly: <?xml version="1.0" encoding="UTF8"?> would be changed to <?xml version="1.0" encoding="ASCII"?>.

To automate access to the CUD REST interface a tool is required that will allow authentication using the GSS Negotiate protocol with a Kerberos service principal keytab. The following tools offer this functionality and have been successfully tested to work with the CUD REST interface.

However, these tools are not the only way to access the CUD REST interface, there is no reason why programmatic solutions should not be developed if you have a requirement to.

curl

The curl tool may be used as long as it has been compiled with GSS-Negotiate and SSL support. Compiling with GSS-Negotiate support is dependent upon the availability of particular Kerberos libraries. The MacOS and Linux operating systems are examples for which curl has been packaged with built-in GSS-Negotiate support, while curl built for the Windows OS may not have been.

To verify whether your version of curl has support for SSL and GSS-Negotiate, invoke curl with the '--version' option, as in the following example:

$ curl --version

An example reply being:

curl 7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
Protocols: dict file ftp ftps http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

If you see "GSS-Negotiate" and "SSL" in the third section of the output ("Features") you can use curl for this.

If the "GSS-Negotiate" (Negotiate authentication) feature is missing you can alternatively use the java-based command-line tool.

NegotiateRestClient.jar (supplied on request)

To use this tool you must have installed the prerequisite Java SE (Standard Edition) Runtime Environment (JRE). This is available as a free download from the official website.  Alternatively an equivalent OpenJDK edition can be installed if it is available for your OS platform. The minimum required Java Runtime Environment (JRE) version is Java 8.

When the JRE is installed the tool can be invoked as follows:

$ java -jar NegotiateRestClient.jar

If a JRE is successfully installed, the output should include an error message, and a usage message resembling the following:

usage: java -jar NegotiateRestClient.jar [options]
  -c,--config <arg> use specified login configuration file
  -D,--debug enables extra debug output, use when verbose error reporting is required
  -d,--data <arg> sends the specified data in a GET request to the HTTP server (required if -f/--upload-file
 not provided). The data will be appended to the URL with a '?' separator.
  -f,--upload-file <arg> streams data from a file and sends the data in a POST request to the HTTP server as
 part of a multipart/formdata request suitable for large files (overrides -d/--data and -f/--file-data)
  -G,--get DEPRECATED
  -h,--help display this message - all other options are ignored if this option is specified
  -k,--krb5-conf use specified krb5.conf file
  -m,--login-module DEPRECATED
  -o,--output-file <arg> output file. If ommitted the output will be send to the console.
  -u,--url <arg> the target URL

Before using the CUD REST interface you should have a Kerberos service principal in the OX.AC.UK realm, which should be authorised appropriately for access to CUD data sets.

Requesting a Service Principal

Please refer to CUD Interfaces - Access

This will enable access to the default CUD Attribute Set to query via REST interface.

Requesting access to CUD data

For authorisation to access to additional CUD attributes, please refer to the information on Requesting access to additional attributes.

Once you have the service principal you can generate a keytab using the kadmin utility on a workstation that has been secured for this purpose (since linux.ox.ac.uk is a shared service it is not recommended for this). Since the keytab contains password-equivalent
keys for system-to-system authentication it should be kept in a secure location.

Using your Keytab

The following examples are based on a fictitious server called example.unit.ox.ac.uk, which is using a service principal, cud/example.unit.ox.ac.uk@OX.AC.UK for which a keytab has been created in the current directory with the name of example.keytab

curl

The following is a simple example of using curl on a Linux platform, authenticating with Kerberos to the CUD REST interface.  Correct Kerberos configuration is assumed.

The Empty Query:

$ KRB5CCNAME=krb5cc_test_cream ; kinit cud/example.unit.ox.ac.uk@OX.AC.UK -k -t example.keytab && curl --negotiate -u : https://ws.cud.ox.ac.uk/cudws/rest/search/person?q=

NegotiateRestClient.jar

In contrast to curl, the java client uses a configuration file in which the principal and the keytab location are specified. Given the example principal and keytab described above we will place our authentication configuration file in the current directory,
and call it login.conf. Its contents will look like the following:

gssnegotiate-keytab-client {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=true
  keyTab="example.keytab"
  principal="cud/example.unit.ox.ac.uk";
};
com.sun.security.jgss.login {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=true
  keyTab="example.keytab"
  principal="cud/example.unit.ox.ac.uk";
};
com.sun.security.jgss.initiate {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=true
  keyTab="example.keytab"
  principal="cud/example.unit.ox.ac.uk";
};
com.sun.security.jgss.accept {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=true
  keyTab="example.keytab"
  principal="cud/example.unit.ox.ac.uk";
};

NOTE that the values assigned to the attributes "keyTab" and "principal" above will need to be changed for your own set-up, i.e. you should substitute the correct values for the location and name of your own keytab file, and the name of your cud service
principal.

With regard to the other configuration attributes, it is important that "useKeyTab" should be set to "true", to ensure that Java uses the keytab file to authenticate to the remote CUD REST interface.

To test the set-up it will help to have a basic kerberos configuration file in place so that the client has all its configuration information available locally. For our example this will be called

krb5.conf

By default the program will look in the current directory for the krb5.conf file. If the file is not found there Java will try to locate this file in these locations on a Windows platform (trying each in turn):

1. %JAVA_HOME%/lib/security/krb5.conf
2. %WINDOWS_ROOT%/krb5.ini

Alternatively you can specify the location of the Kerberos configuration file on the java command line, by setting the java.security.krb5.conf parameter, as in

% java -Djava.security.krb5.conf=/path/to/krb5.conf ...(etc)...

For use with the Oxford realm the contents of the krb5.conf kerberos configuration file should look like the following:

[libdefaults]
default_realm = OX.AC.UK
[realms]
OX.AC.UK = {
  kdc = kdc0.ox.ac.uk
  kdc = kdc1.ox.ac.uk
  kdc = kdc2.ox.ac.uk
  kdc = kdc3.ox.ac.uk
  admin_server = kdc-admin.ox.ac.uk
  default_domain = OX.AC.UK
}
[domain_realm]
.OX.AC.UK = OX.AC.UK

The krb5.conf above can be re-used as-is.

So to summarise, for this example we should have three files available: the keytab file (example.keytab), the authentication configuration file (login.conf), and the kerberos configuration file (krb5.conf). With that in mind the following are examples
of using NegotiateRestClient.jar, and authenticating transparently with Kerberos to the CUD REST interface.

The Empty Query

The query is specified using the -d/--data flag, and the arguments following will be appended to the URL in a GET request. In this case, "q=" indicates that the query string is empty.

% java -Djava.security.krb5.conf=krb5.conf -jar NegotiateRestClient.jar -c login.conf -u https://ws.cud.ox.ac.uk/cudws/rest/search/person -d "q="

Some Example Queries

The following query retrieves all records that have a currently-active affiliation to OUCS, in XML format:

% java -Djava.security.krb5.conf=krb5.conf -jar NegotiateRestClient.jar -c login.conf -u https://ws.cud.ox.ac.uk/cudws/rest/search/person -d "q=cud\:cas\:current_affiliation:oucs&format=xml"

The next query retrieves a particular record for a person, with change history, in json format:

% java -Djava.security.krb5.conf=krb5.conf -jar NegotiateRestClient.jar -c login.conf -u https://ws.cud.ox.ac.uk/cudws/rest/search/person -d "q=cud\:cas\:sso_username:unit1234&format=json&history=y"