Codedrop™ Weblog

This past week has left me having to learn much more about https certificates then I ever thought I would care to know…   Here’s a synopsis of some of the highlights of my learnings as pulled from various resources on the web.

OpenSSL supports several certificate formats. Certificates are based on the DSA signature algorithm and the RSA algorithm for public-key cryptography according to PKCS algorithms.  The certificate format depends on the application, as there is no agreement on file format standards.

Private keys are usually available in the PEM and DER format. The related files have names of the following type:

*key-rsa.pem for pem files
*key-rsa.der for der files

For OpenSSL applications, the PEM format should suffice. For Java applications, the DER format might be more suitable for importing the private key and certificates.

For certificates, the available formats are PEM, DER and PKCS12 with file names of the following type:

*cert.pem for pem files
*cert.der for der files
*cert.p12 for pkcs12 files

In general, the PEM formats are mostly used in the Unix world, PCKS12 in the Microsoft world and DER in the Java world.

Certificate files are ASN.1-encoded objects that may be encrypted according to DES (Data Encryption Standard). The files can optionally be encrypted using a symmetric cipher algorithm, such as 3DES.

An unencrypted PEM file might look something like this:

    —–BEGIN CERTIFICATE—–
    MB4CGQDUoLoCULb9LsYm5+/WN992xxbiLQlEuIsCAQM=
    —–END CERTIFICATE—–

The string beginning with MB4C… is the Base64-encoded, ASN.1-encoded object.

An encrypted file would have headers describing the type of encryption used, and the initialization vector:

    —–BEGIN RSA PRIVATE KEY—–
    Proc-Type: 4,ENCRYPTED
    DEK-Info: DES-EDE3-CBC,C814158661DC1449
    AFAZFbnQNrGjZJ/ZemdVSoZa3HWujxZuvBHzHNoesxeyqqidFvnydA==
    —–END RSA PRIVATE KEY—–

The two headers Proc-Type and DEK-Info declare the type of encryption, and the string starting with AFAZ… is the Base64-encoded, encrypted, ASN.1-encoded object.

As web browsers make use of Java applications, they import/export certificates in pkcs12 file format, i.e. public and private keys are packed in one single file using the PKCS#12 algorithm. Other applications require the pem format with unpacked public and private keys, thus the user must remember the appropriate file format for each application and must perform format conversions as appropriate.

The following tables report a summary of formats used for INFN-Grid applications and two simple scripts with format conversion commands.

INFN-Grid Certificates Format Summary
Certificate Type     Certificate Format
CA Authority Certificate     DER
Personal Certificate from CA     PKCS12
Grid Access Certificate     PEM

=========================
CONVERT pkcs12 to pem
=========================
#!/bin/sh
echo “copy your cert to cert.p12 – then run this script”
openssl pkcs12 -clcerts -nokeys -in cert.p12 -out usercert.pem
openssl pkcs12 -nocerts -in cert.p12 -out userkey.pem

=========================
CONVERT pem to pkcs12
=========================
#!/bin/sh
echo “Verify that you are using the correct certificate pair (key/cert)”
openssl pkcs12 -export -out one.identity.neteller.com.p12 -inkey ./one.identity.neteller.com.key -in ./one.identity.neteller.com.cert

** NOTE: specify the -in and -inkey parameters as PEM format files…

If your running JRockit, you might also be interested in how to update Verisign CA root certificates.

This might be required if you start seeing errors such as this:
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

After weeks of frustration with my RHEL terminal windows closing themselves after a small interval of inactivity, (go for lunch and my terminals were closed)… I finally found out the solution to prevent this ‘default’ behaviour that comes in RHEL.

The /etc/profile was the culprit.  Simply remove the following lines or update them to a much more respectable time interval and restart your X windows.

TMOUT=3600
export TMOUT

Thats one I’ll not forget as its sure a pain when your in development mode with all your terminals set where you want them…. turn away for an hour and find them all closed!

A very useful feature of Oracle is the ‘CONNECT BY’ command. I make use of this whenever I need to generate any SQL output that has any sequential data as a key to the query. For example, a report of the number of logins per day or per hour.

To use connect by in your query, simply add a block to the ‘from’ clause section of you query and then reference its values the way you would any other table.

Here’s a few examples that return a sequential range of date/times based on current sysdate. Whats nice about this is that the sysdate is a moving target so you data is always kept up to date!

select to_char(x.lvl, 'YYYY-MM-DD HH24') || ':00'
from (  SELECT sysdate - (level/24)  lvl
      	FROM dual
      	CONNECT BY LEVEL <= 24 ) x

Outputs:
2009-08-19 14:00,
2009-08-19 13:00,
2009-08-19 12:00,
2009-08-19 11:00,
2009-08-19 10:00

select to_char(x.lvl, 'YYYY-MM-DD HH24') || ':00'
from (  SELECT sysdate - (12*level/24) lvl
      	FROM dual
      	CONNECT BY LEVEL <= 30 ) x

Outputs:
2009-08-19 03:00,
2009-08-18 15:00,
2009-08-18 03:00,
2009-08-17 15:00,
2009-08-17 03:00

select to_char(x.lvl, 'YYYY-MM-DD')
from ( SELECT sysdate - level lvl
      	FROM dual
      	CONNECT BY LEVEL <= 30) x

Outputs:
2009-08-17,
2009-08-16,
2009-08-15,
2009-08-14,
2009-08-13,
etc...

I keep coming across instances of ant build files where developers have hard coded a usage target that outputs target actions.  This is not required and should be avoided as it is often not maintained.  Instead, properly maintain the description attribute of each target and use a default target as below:  

<project name="foo" basedir="." default="usage">

...

<target name="usage">
        <java classname="org.apache.tools.ant.Main">
            <arg value="-projecthelp"/>
            <classpath>
                <pathelement location="${lib.dir}/ant/ant.jar"/>
                <pathelement location="${lib.dir}/ant/ant-launcher.jar"/>
                <pathelement location="${lib.dir}/ant/xerces_2.8.0.jar"/>
            </classpath>
        </java>
    </target>

...

</project>

Trying to identify all modules installed on your local server?… try this command:
Note: First time you run you might have to configure CPAN… for the most part defaults are good but pick a good local mirror.

perl -MCPAN -e 'print CPAN::Shell->r '

Package namespace    installed    latest  in CPAN file
Archive::Tar              1.30      1.52  K/KA/KANE/Archive-Tar-1.52.tar.gz
Attribute::Handlers    0.78_01      0.85  S/SM/SMUELLER/Attribute-Handlers-0.85.tar.gz
AutoLoader                5.60      5.68  S/SM/SMUELLER/AutoLoader-5.68.tar.gz
B                         1.02      1.19  N/NW/NWCLARK/perl-5.8.9.tar.gz
B::Debug                  1.01      1.11  R/RU/RURBAN/B-Debug-1.11.tar.gz

Grails application throwing OutOfMemory error and PermGen space was exceeeded errors… Looks like grails applications require a fairly large permgen space… Here’s a useful  article on how to address this.

Here’s a quick solution for trying to extract the raw xml string from an Oracle XDB database when working with Grails.

After an initial attempt to hardcode a hibernate query to return the raw sql using .getStringVal() as follows:

  SELECT x.id, x.xmlData xmldata.getStringVal(), lines.*
  FROM xml_requests x
       XMLTable('declare default element namespace
				"http://www.foo.com/fooservice";
                for $i in /XXXResponse/ResponseData/*/node()
                where $i/*:Error/*:Code = "0"
                return $i'
                PASSING x.xmldata
                COLUMNS GivenName NVARCHAR2(200)
	          PATH './*:PersonAdresseData/*:Person/*:Navn/*:Fornavne',
                    Surname  NVARCHAR2(200)
	          PATH './*:PersonAdresseData/*:Person/*:Navn/*:Efternavn') lines
  WHERE ....

I found that the query would blow up if the xml string returned exceeded some threshold.  Approx ~ 2000 chars as I’m using an oracle database.

Solution was relatively simple thanks to the various support libraries Oracle brings to the table.

Add the xdb.jar and xmlparserv2.jar libraries from the Oracle installation into your Grails application /lib directory and then modify the .gsp page to handle the custom type accordingly as below:

  <div class="list">
    <table>
      <thead>
      <tr>
        <g:each var="key" in="${list[0].keySet()}">
          <g:sortableColumn property="${key}" title="${key}"/>
        </g:each>
      </tr>
      </thead>
      <tbody>
      <g:each in="${list}" status="i" var="item">
        <tr class="${(i % 2) == 0 ? 'odd' : 'even'}">
          <g:each var="key" in="${list[0].keySet()}">
            <td>
            <g:if test="${item[key] instanceof oracle.xdb.XMLType}">
              ${item[key].getStringVal()}
            </g:if>
            <g:else>
              ${item[key]}
            </g:else>
            </td>
          </g:each>
        </tr>
      </g:each>
      </tbody>
    </table>
  </div>

** The above code will dynamically render a table view using the column names as headers in an HTML table. In my eaxmple one the columns I wanted to display was XMLType and stored in Oracle XDB format.

When javac is compiling a large number of java source files, it may fail with java.lang.OutOfMemoryError:

The system is out of resources.
Consult the following stack trace for details.
java.lang.OutOfMemoryError: Java heap space

It’s no different than OutOfMemoryError in other java applications. When you run javac in Sun JDK, it’s invoking com.sun.tools.javac.main.Main located in %JAVA_HOME%\lib\tools.jar.

If you are compiling with javac task in Apache Ant, set fork attribute to true, to run javac in a separate process with its own heap size settings. If fork is set to false, or not set (default is false), javac will run in the same process as Ant, which has a default maximum heap size of 64m.

<javac fork="true">
       srcdir="${basedir}/src"
       destdir="${basedir}/build/classes"
       classpath="${project.classpath}"
       memoryinitialsize="256m"
       memorymaximumsize="256m"&gt;
</javac>

Setting fork to true will also limit any memory leaks in javac implementation to its own child process, without affecting the parent Ant process.

Looking for a good tool for testing SOAP web services?…soapUI to the rescue!

What exactly is soapUI?
soapUI is the leading desktop application for inspecting, invoking,
monitoring, simulating/mocking and
functional/load/compliance/surveillance testing of REST/WADL and
SOAP/WSDL-based Web Services over HTTP.

There are plugins Eclipse, IntelliJ, NetBeans and Maven.   I’ve tried the plugin using IntelliJ 8.1.1 and it works like a charm!  This is an invaluable tool if your actively developing SOAP apis!

You don’t have to be much of a Techy to know about the power Firebug brings to the Firefox browser.  I’ve often been frustrated but the lack thereof of such support in IE… until now.  The Internet Explorer (IE) Developer Toolbar has been around for quite some time now, but in my ignorance I had lost track of it…  so here it is again for the world to use… Like Firebug it brings in a panel bar from which you can quickly navigate page elements, highlight divs, tables, etc…