Codedrop™ Weblog

Gmail’s great… and who doesn’t have a gmail account nowadays. One thing I’ve always wondered was how to search for all unread messages, and not just those on the current page?… Who wants to seatch backwards page by page looking for unread messages anyways…

Well after years of using it today I finally found the answer. Simply search for ‘label:unread‘ messages and your set!.. wonder what other nifty searches you can preform?

Tweet This Post

Setting up trac on dreamhost isn’t a very well documented process.  The one-click install does setup a project, but your left with bare bones install and no help on getting the initial admin account created.  I found the dreamy-trac installation script worked well.  Beyond this and the basic config changes you have to do I found the following settings are required in your trac.ini file in order to get your email notifications working on Dreamhost when you use GMail as your mail provider.

[notification]
smtp_enabled = true
use_tls = true
mime_encoding = base64
smtp_server = smtp.gmail.com
smtp_port = 587
smtp_user = user
smtp_password = password

Tweet This Post

Here’s a nice use of linux find to locate all ssl certificate files on a filesystem stored in directories name /certificates that excludes pesky .svn results:


find . -path "*/certificates/*" -not \( -name .svn -prune \)

I recently used this technique for pulling all certificates that were scatterred throughout a svn repository. Taking this output you can then easily copy them to another location (say for production deployment)

find . -path "*/certificates/*" -not \( -name .svn -prune \) -exec cp '{}' ./deploy/dist/certificates ';'

To go one step further integrating this into an ANT build proved to be a pain in the butt.  The ant exec task was causing me alot of grief so ended up using antcontrib tasks like this:


<target name="distCerts"
description="Prepare folder with all certificates required for deployment.">
<delete  dir="${projectRoot}/deploy/dist/certificates"/>
<mkdir dir="${projectRoot}/deploy/dist/certificates"/>
<shellscript shell="sh" dir="${projectRoot}">
find ./servicegroup -path "*/certificates/*" -not \( -name .svn -prune \) -exec cp '{}' ./deploy/dist/certificates ';'
find ./deploy/certificates -path "*" -not \( -name .svn -prune \) -exec cp '{}' ./deploy/dist/certificates ';'
</shellscript>
</target>


… much easier!

Tweet This Post

3 days wasted trying to solve a TIBCO issue with EMS messages. 

Heres the scenario:

Utilizing a TIBCO service with a “JMS Queue Requestor” that communicated with a backoffice java service (spring, jboss rules).  The TIBCO service published messages to a specific ems queue.. MYSERVICE.REQUEST and would listen for responses to queue MYSERVICE.RESPONSE.  The backend service would pick up messages from the MYSERVICE.REQUEST queue and write the response back to MYSERVICE.RESPONSE.  Fairly standard flow here.

We started making use of a static reply queue and adding correlation_id on the JMS request msg so that random temporary queues would not be created (easier for debugging) for the response.  Suddenly we received random occurrences where  the message response no longer matched the associated request.  I was able to easily recreate this problem in a development environment by creating a perl program that made several simultaneous requests to our service on different threads.  With as few a 2 threads errors started occuring all the time.

After some time going through our custom java code I ended up utilizing GEMS to monitor the EMS queues incoming/outgoing responses.  All looked fine here… so that pointed back at TIBCO.  Thus leading me to eventually stumble across the solution in the TIBCO support forums.  The solution is posted below but I find it Interesting that BW Designer interface allows you to create this scenario in the first place.  Seems like a bug to me….  This ‘feature’ could easily cost a business many days and could be a critical failure point if it made its way into production systems.. imagine a financial application where messages were getting mixed up.  Surely there can be more controls to prevent this scenario with the development tools.

** This highlights an important point to all you IT decision makers out there.  Don’t just buy into the marketing/sales pitch when investigating software alternatives… Get valid references and speak to people using the product to get their true feedback on the product.

How to correlate EMS messages in a request response scenario

Case1

Consider the scenario where you are using a JMS Queue Requestor which sends a request and waits for a reply. Additionally, you have a corresponding process (say a JMSQueue receiver) which receives these requests and sends back replies (Reply To JMS Message).

JMS request/reply activity uses temporary destinations to ensure that reply messages are received only by the process instance that sent the request. While sending each request the JMS Queue requestor creates a temp queue for the reply. It then sends the temp reply queue name along with the request message. The temporary queue name is unique for each process instance.

If the replyToQueue queue (static) is specified then all replies will be sent to the same queue and there will be no guarantee that the correct reply will be received by the process instance that sent the request.

You can use an expression for the replyToQueue to create different replyToDestinations for each request.

Case2

In Case1, if you need to use constant destinations for all replies and you do not want to use temporary destinations, then proceed with the following. Instead of using JMSQueueRequestor you can use the pair of JMSQueueSender and Wait for JMSQueueMessage Activities and then map the messageID of the JMSSender as the event key of the Wait for JMS activity and use the JMSCorrelationID header of the input message as the Candidate Event Key.

Tweet This Post

Running the TIBCO stack on a RHEL 4.7 environment I recently hit a problem where uploading a new .ear file through the Adminstrator console would error and render a blank page.  Some digging into the logs yielded an error when trying to write to a temporary folder that was under /tmp.  Restarting the entire system would re-create the new directory under /tmp but after a few days it dissappeared (o/s cleaned it up) and the error re-occurred.

Setting the java.property.java.io.tmpdir environment variable acts as a permanent fix for this as you can specify a custom directory that will not be purged automatically.

Modify these config files:

<tibco_home>/administrator/domain/<domain_name>/bin/tibcoadmin_<domain_name>.tra
<tibco_home>/tra/domain/<domain_name>/hawkagent_<domain_name>.tra

To add the following:

# CUSTOM – Specify the location of the temporary files for upload.  System default of
# /tmp will error after a few days as the dirs get cleaned out.
java.property.java.io.tmpdir=<path>

ie. java.property.java.io.tmpdir=/opt/tibcotmp

** NOTE: Using a custom token in your comment (ie: CUSTOM) will allow easy identification of all config files that you modified you add a custom property that is outside the normal configuration.

Tweet This Post

Found a great article at carbonfive.com on how to configure application with Spring utilizing various techniques.  My favorite from this article is included below:

Optional External Properties

There’s another use case that applies to some projects. Often in non-developer environments, system admins want to keep properties for the environment outside of the deployable archive or the application server, and they don’t want to deal with keeping those files in a Tomcat context file; they prefer a simple properties file. They also don’t want to have to place the file in a hard-coded location (e.g. /var/acmeapp/application.properties) or they may keep configuration for multiple servers in the same network directory, each file names after the server. With a little trickery, it’s easy to support an optional external properties file that isn’t in a hard-coded location. The location of the file is passed as a single system property to the JVM, for example: -Dconfig=file://var/acmeapp/server1.properties.

Here’s the configuration to make it happen:

<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="ignoreUnresolvablePlaceholders" value="true">
</property>

<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="ignoreResourceNotFound" value="true">
    <property name="location" value="${config}">
</property>

</property></bean></bean>

The first definition enables basic property resolution through system properties (in fallback mode). The second bean loads the resource from the location resolved from the system property -Dconfig. All spring resource urls are supported, making this very flexible.

Best Practices

• Deploy the same exact artifact (e.g. war, ear, etc) across all environments by externalizing configuration. This may seem daunting, but the emergent benefits are huge in terms of simplicity.
• Only make things that can safely change across environments configurable. Also, only things that need to be configurable should be configurable, it’s easy to go overboard.
• Configure the minimal properties search path that meets your requirements.
• When looking for properties files in the project tree, use classpath resources whenever possible. This makes finding those files easy, consistent, and insensitive to the working-dir, which is great when running tests from your IDE and command line.
• Aim for a zero-configuration check-out, build, run-tests cycle for the environment where its happens most: development.

Tweet This Post

This past week has left me having to learn much more about https certificates then I ever thought I would care to know…   Here’s a synopsis of some of the highlights of my learnings as pulled from various resources on the web.

OpenSSL supports several certificate formats. Certificates are based on the DSA signature algorithm and the RSA algorithm for public-key cryptography according to PKCS algorithms.  The certificate format depends on the application, as there is no agreement on file format standards.

Private keys are usually available in the PEM and DER format. The related files have names of the following type:

*key-rsa.pem for pem files
*key-rsa.der for der files

For OpenSSL applications, the PEM format should suffice. For Java applications, the DER format might be more suitable for importing the private key and certificates.

For certificates, the available formats are PEM, DER and PKCS12 with file names of the following type:

*cert.pem for pem files
*cert.der for der files
*cert.p12 for pkcs12 files

In general, the PEM formats are mostly used in the Unix world, PCKS12 in the Microsoft world and DER in the Java world.

Certificate files are ASN.1-encoded objects that may be encrypted according to DES (Data Encryption Standard). The files can optionally be encrypted using a symmetric cipher algorithm, such as 3DES.

An unencrypted PEM file might look something like this:

    —–BEGIN CERTIFICATE—–
    MB4CGQDUoLoCULb9LsYm5+/WN992xxbiLQlEuIsCAQM=
    —–END CERTIFICATE—–

The string beginning with MB4C… is the Base64-encoded, ASN.1-encoded object.

An encrypted file would have headers describing the type of encryption used, and the initialization vector:

    —–BEGIN RSA PRIVATE KEY—–
    Proc-Type: 4,ENCRYPTED
    DEK-Info: DES-EDE3-CBC,C814158661DC1449
    AFAZFbnQNrGjZJ/ZemdVSoZa3HWujxZuvBHzHNoesxeyqqidFvnydA==
    —–END RSA PRIVATE KEY—–

The two headers Proc-Type and DEK-Info declare the type of encryption, and the string starting with AFAZ… is the Base64-encoded, encrypted, ASN.1-encoded object.

As web browsers make use of Java applications, they import/export certificates in pkcs12 file format, i.e. public and private keys are packed in one single file using the PKCS#12 algorithm. Other applications require the pem format with unpacked public and private keys, thus the user must remember the appropriate file format for each application and must perform format conversions as appropriate.

The following tables report a summary of formats used for INFN-Grid applications and two simple scripts with format conversion commands.

INFN-Grid Certificates Format Summary
Certificate Type     Certificate Format
CA Authority Certificate     DER
Personal Certificate from CA     PKCS12
Grid Access Certificate     PEM

=========================
CONVERT pkcs12 to pem
=========================
#!/bin/sh
echo “copy your cert to cert.p12 – then run this script”
openssl pkcs12 -clcerts -nokeys -in cert.p12 -out usercert.pem
openssl pkcs12 -nocerts -in cert.p12 -out userkey.pem

=========================
CONVERT pem to pkcs12
=========================
#!/bin/sh
echo “Verify that you are using the correct certificate pair (key/cert)”
openssl pkcs12 -export -out one.identity.neteller.com.p12 -inkey ./one.identity.neteller.com.key -in ./one.identity.neteller.com.cert

** NOTE: specify the -in and -inkey parameters as PEM format files…

If your running JRockit, you might also be interested in how to update Verisign CA root certificates.

This might be required if you start seeing errors such as this:
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Tweet This Post

After weeks of frustration with my RHEL terminal windows closing themselves after a small interval of inactivity, (go for lunch and my terminals were closed)… I finally found out the solution to prevent this ‘default’ behaviour that comes in RHEL.

The /etc/profile was the culprit.  Simply remove the following lines or update them to a much more respectable time interval and restart your X windows.

TMOUT=3600
export TMOUT

Thats one I’ll not forget as its sure a pain when your in development mode with all your terminals set where you want them…. turn away for an hour and find them all closed!

Tweet This Post

A very useful feature of Oracle is the ‘CONNECT BY’ command. I make use of this whenever I need to generate any SQL output that has any sequential data as a key to the query. For example, a report of the number of logins per day or per hour.

To use connect by in your query, simply add a block to the ‘from’ clause section of you query and then reference its values the way you would any other table.

Here’s a few examples that return a sequential range of date/times based on current sysdate. Whats nice about this is that the sysdate is a moving target so you data is always kept up to date!

select to_char(x.lvl, 'YYYY-MM-DD HH24') || ':00'
from (  SELECT sysdate - (level/24)  lvl
      	FROM dual
      	CONNECT BY LEVEL <= 24 ) x

Outputs:
2009-08-19 14:00,
2009-08-19 13:00,
2009-08-19 12:00,
2009-08-19 11:00,
2009-08-19 10:00

select to_char(x.lvl, 'YYYY-MM-DD HH24') || ':00'
from (  SELECT sysdate - (12*level/24) lvl
      	FROM dual
      	CONNECT BY LEVEL <= 30 ) x

Outputs:
2009-08-19 03:00,
2009-08-18 15:00,
2009-08-18 03:00,
2009-08-17 15:00,
2009-08-17 03:00

select to_char(x.lvl, 'YYYY-MM-DD')
from ( SELECT sysdate - level lvl
      	FROM dual
      	CONNECT BY LEVEL <= 30) x

Outputs:
2009-08-17,
2009-08-16,
2009-08-15,
2009-08-14,
2009-08-13,
etc...

Tweet This Post

I keep coming across instances of ant build files where developers have hard coded a usage target that outputs target actions.  This is not required and should be avoided as it is often not maintained.  Instead, properly maintain the description attribute of each target and use a default target as below:  

<project name="foo" basedir="." default="usage">

...

<target name="usage">
        <java classname="org.apache.tools.ant.Main">
            <arg value="-projecthelp"/>
            <classpath>
                <pathelement location="${lib.dir}/ant/ant.jar"/>
                <pathelement location="${lib.dir}/ant/ant-launcher.jar"/>
                <pathelement location="${lib.dir}/ant/xerces_2.8.0.jar"/>
            </classpath>
        </java>
    </target>

...

</project>

Tweet This Post