Improve Error Reporting in Java EE

9 March 2015

While I am a great advocate of adopting standards and have come to dislike ducttaped together solutions of the years, I still have some qualms with Java (EE) in general and WildFly in particular.

My number one dislike is the error reporting during deployment. These are inscrutable, unsolvable, unhelpful messages that are complete and utter gibberish. If there should be one fricking standard that should be adopted for Java EE 8 it should be better, standardized error reporting.

Exhibit A:

14:47:37,980 ERROR [] (MSC service thread 1-15) MSC000001: Failed to start service jboss.deployment.subunit."eduarte-deliverable-dev-ear.ear"."eduarte-common-dao-2.42-SNAPSHOT.jar".PARSE: org.jboss.msc.service.StartException in service jboss.deployment.subunit."eduarte-deliverable-dev-ear.ear"."eduarte-common-dao-2.42-SNAPSHOT.jar".PARSE: JBAS018733: Failed to process phase PARSE of subdeployment "eduarte-common-dao-2.42-SNAPSHOT.jar" of deployment "eduarte-deliverable-dev-ear.ear"
	at [wildfly-server-8.1.0.Final.jar:8.1.0.Final]
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService( [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
	at org.jboss.msc.service.ServiceControllerImpl$ [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
	at java.util.concurrent.ThreadPoolExecutor.runWorker( [jrebel-bootstrap-a53fd962a83484078782af30d2cabc06.jar:1.8.0_31]
	at java.util.concurrent.ThreadPoolExecutor$ [jrebel-bootstrap-a53fd962a83484078782af30d2cabc06.jar:1.8.0_31]
	at [jrebel-bootstrap-a53fd962a83484078782af30d2cabc06.jar:]
Caused by: org.jboss.msc.service.ServiceNotFoundException: Service service jboss.ejb.default-resource-adapter-name-service not found
	at org.jboss.msc.service.ServiceContainerImpl.getRequiredService( [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
	at [wildfly-server-8.1.0.Final.jar:8.1.0.Final]
	... 5 more

And a couple of hundred lines below:

14:47:42,608 ERROR [] (Controller Boot Thread) JBAS014613: Operation ("deploy") failed - address: ([("deployment" => "eduarte-deliverable-dev-ear.ear")]) - failure description: {"JBAS014671: Failed services" => {"jboss.deployment.subunit.\"eduarte-deliverable-dev-ear.ear\".\"eduarte-common-dao-2.42-SNAPSHOT.jar\".PARSE" => "org.jboss.msc.service.StartException in service jboss.deployment.subunit.\"eduarte-deliverable-dev-ear.ear\".\"eduarte-common-dao-2.42-SNAPSHOT.jar\".PARSE: JBAS018733: Failed to process phase PARSE of subdeployment \"eduarte-common-dao-2.42-SNAPSHOT.jar\" of deployment \"eduarte-deliverable-dev-ear.ear\"
    Caused by: org.jboss.msc.service.ServiceNotFoundException: Service service jboss.ejb.default-resource-adapter-name-service not found"}}
14:47:42,642 INFO  [] (ServerService Thread Pool -- 28) JBAS018559: Deployed "eduarte-portal-eo-student.war" (runtime-name : "eduarte-portal-eo-student.war")
14:47:42,642 INFO  [] (ServerService Thread Pool -- 28) JBAS018559: Deployed "eduarte-portal-eo-ouder.war" (runtime-name : "eduarte-portal-eo-ouder.war")
14:47:42,642 INFO  [] (ServerService Thread Pool -- 28) JBAS018559: Deployed "eduarte-portal-eo-docent.war" (runtime-name : "eduarte-portal-eo-docent.war")
14:47:42,642 INFO  [] (ServerService Thread Pool -- 28) JBAS018559: Deployed "eduarte-portal-eo-bedrijf.war" (runtime-name : "eduarte-portal-eo-bedrijf.war")
14:47:42,643 INFO  [] (ServerService Thread Pool -- 28) JBAS018559: Deployed "eduarte-portal-eo-authenticator.war" (runtime-name : "eduarte-portal-eo-authenticator.war")
14:47:42,643 INFO  [] (ServerService Thread Pool -- 28) JBAS018559: Deployed "eduarte-deliverable-dev-ear.ear" (runtime-name : "eduarte-deliverable-dev-ear.ear")
14:47:42,644 INFO  [] (Controller Boot Thread) JBAS014774: Service status report
JBAS014777:   Services which failed to start:      service jboss.deployment.subunit."eduarte-deliverable-dev-ear.ear"."eduarte-common-dao-2.42-SNAPSHOT.jar".PARSE: org.jboss.msc.service.StartException in service jboss.deployment.subunit."eduarte-deliverable-dev-ear.ear"."eduarte-common-dao-2.42-SNAPSHOT.jar".PARSE: JBAS018733: Failed to process phase PARSE of subdeployment "eduarte-common-dao-2.42-SNAPSHOT.jar" of deployment "eduarte-deliverable-dev-ear.ear"

14:47:42,653 INFO  [] (Controller Boot Thread) JBAS015961: Http management interface listening on
14:47:42,653 INFO  [] (Controller Boot Thread) JBAS015951: Admin console listening on
14:47:42,654 ERROR [] (Controller Boot Thread) JBAS015875: WildFly 8.1.0.Final "Kenny" started (with errors) in 22967ms - Started 1208 of 1315 services (4 services failed or missing dependencies, 225 services are lazy, passive or on-demand)
14:47:42,936 INFO  [] (MSC service thread 1-13) JBAS015974: Stopped subdeployment (runtime-name: eduarte-common-dao-2.42-SNAPSHOT.jar) in 38ms
14:47:42,942 INFO  [] (MSC service thread 1-7) JBAS015974: Stopped subdeployment (runtime-name: eduarte-ws-eo-rest-2.42-SNAPSHOT.war) in 44ms
14:47:43,054 INFO  [] (MSC service thread 1-6) JBAS015974: Stopped subdeployment (runtime-name: eduarte-sis-web-main.war) in 156ms
14:47:43,088 INFO  [] (MSC service thread 1-9) JBAS015877: Stopped deployment eduarte-deliverable-dev-ear.ear (runtime-name: eduarte-deliverable-dev-ear.ear) in 195ms
14:47:43,136 INFO  [] (DeploymentScanner-threads - 2) JBAS018558: Undeployed "eduarte-deliverable-dev-ear.ear" (runtime-name: "eduarte-deliverable-dev-ear.ear")
14:47:43,137 INFO  [] (DeploymentScanner-threads - 2) JBAS014774: Service status report
JBAS014777:   Services which failed to start:      service jboss.deployment.subunit."eduarte-deliverable-dev-ear.ear"."eduarte-common-dao-2.42-SNAPSHOT.jar".PARSE

14:47:47,715 WARN  [] (DeploymentScanner-threads - 2) JBAS015002: Deployment of 'eduarte-web-main.war' requested, but the deployment is not present
14:47:47,715 INFO  [] (DeploymentScanner-threads - 2) JBAS015003: Found eduarte-deliverable-dev-ear.ear in deployment directory. To trigger deployment create a file called eduarte-deliverable-dev-ear.ear.dodeploy

Apparently something is missing, but the stack traces don’t contain any code that is under my purview, and the error message is really clear and helpful in identifying what is missing:

MSC000001: Failed to start service jboss.deployment.subunit.”eduarte-deliverable-dev-ear.ear”.”eduarte-common-dao-2.42-SNAPSHOT.jar”. PARSE: org.jboss.msc.service.StartException in service jboss.deployment.subunit.”eduarte-deliverable-dev-ear.ear”.”eduarte-common-dao-2.42-SNAPSHOT.jar”. PARSE: JBAS018733: Failed to process phase PARSE of subdeployment “eduarte-common-dao-2.42-SNAPSHOT.jar” of deployment “eduarte-deliverable-dev-ear.ear”

Which is caused by:

org.jboss.msc.service.ServiceNotFoundException: Service service jboss.ejb.default-resource-adapter-name-service not found

What the hell is Wildfly trying to tell me?

The solution (of course) is to run the configuration script that is external to our application, in order to tell Wildfly that a new message queue should be added:

15:03:30,952 WARN  [org.jboss.messaging] (management-handler-thread - 2) JBAS011618: There is no resource matching the expiry-address jms.queue.ExpiryQueue for the address-settings #, expired messages from destinations matching this address-setting will be lost!
15:03:30,952 WARN  [org.jboss.messaging] (management-handler-thread - 2) JBAS011619: There is no resource matching the dead-letter-address jms.queue.DLQ for the address-settings #, undelivered messages from destinations matching this address-setting will be lost!
15:03:31,033 WARN  [] (MSC service thread 1-16) JBAS011600: AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal
15:03:31,149 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 72) HQ221000: live server is starting with configuration HornetQ Configuration (clustered=false,backup=false,sharedStore=true,journalDirectory=/Users/dashorst/Workspaces/luna/wildfly-8.1.0.Final/standalone/data/messagingjournal,bindingsDirectory=/Users/dashorst/Workspaces/luna/wildfly-8.1.0.Final/standalone/data/messagingbindings,largeMessagesDirectory=/Users/dashorst/Workspaces/luna/wildfly-8.1.0.Final/standalone/data/messaginglargemessages,pagingDirectory=/Users/dashorst/Workspaces/luna/wildfly-8.1.0.Final/standalone/data/messagingpaging)
15:03:31,151 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 72) HQ221006: Waiting to obtain live lock
15:03:31,222 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 72) HQ221013: Using NIO Journal
15:03:31,288 INFO  [io.netty.util.internal.PlatformDependent] (ServerService Thread Pool -- 72) Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system unstability.
15:03:31,359 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 72) HQ221043: Adding protocol support CORE
15:03:31,372 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 72) HQ221043: Adding protocol support AMQP
15:03:31,380 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 72) HQ221043: Adding protocol support STOMP
15:03:31,427 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 72) HQ221034: Waiting to obtain live lock
15:03:31,428 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 72) HQ221035: Live Server Obtained live lock
15:03:31,610 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 72) HQ221007: Server is now live
15:03:31,610 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 72) HQ221001: HornetQ Server version 2.4.1.Final (Fast Hornet, 124) [1141831d-c665-11e4-86db-0ffb92fb33f7] 
15:03:31,666 INFO  [] (ServerService Thread Pool -- 72) JBAS011601: Bound messaging object to jndi name java:/messaging/ConnectionFactory
15:03:31,672 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 73) HQ221003: trying to deploy queue jms.queue.eventQueue
15:03:31,785 INFO  [] (ServerService Thread Pool -- 73) JBAS011601: Bound messaging object to jndi name queue/eventQueue
15:03:31,820 INFO  [] (MSC service thread 1-10) JBAS010406: Registered connection factory java:/messaging/JmsXA
15:03:31,908 INFO  [org.hornetq.ra] (MSC service thread 1-10) HornetQ resource adaptor started
15:03:31,908 INFO  [$ResourceAdapterActivator] (MSC service thread 1-10) IJ020002: Deployed: file://RaActivatorhornetq-ra
15:03:31,910 INFO  [] (MSC service thread 1-5) JBAS010401: Bound JCA ConnectionFactory [java:/messaging/JmsXA]
15:03:31,910 INFO  [] (MSC service thread 1-16) JBAS011601: Bound messaging object to jndi name java:jboss/DefaultJMSConnectionFactory

Unfortunately the original error message did not provide any useful insights as to what was missing.

20 February 2015

A list of all active space probe missions including mission details.

New Horizons Rosetta
The New Horizons and Rosetta missions

Via @kottke

Brianna Wu Risking Life versus Gamergate

20 February 2015

Unreal what happens to Brianna Wu. I admire her will to continue to fight the good fight and I hope she’ll be safe. Nobody deserves what she has to go through. This is from 9 days ago:

This weekend, a man wearing a skull mask posted a video on YouTube outlining his plans to murder me. I know his real name. I documented it and sent it to law enforcement, praying something is finally done. I have received these death threats and 43 others in the last five months.

Read the whole article and be appalled at the threats and vitriol she recieves for being a woman in tech.

IBM Design Language

10 February 2015

Who’d thought: IBM talking design.

“Living Language – A shared vocabulary for design”

Via Daring Fireball

Designing a Backend for a Famous Backend

10 February 2015

When Kim Kardashian and Paper magazine set out to break the internet, it was the task of one engineer to make sure that Paper’s backend systems wouldn’t crumble under the weight of Kim’s backend.

Hosting that butt is an impressive feat. You can’t just put Kim Kardashian nudes on the Internet and walk away —that would be like putting up a tent in the middle of a hurricane. Your web server would melt. You need to plan.

And the understatement of the year:

“We may get a lot of traffic.”

Rather interesting read how to scale a system with mostly static content, but that will get hammered by millions within a few hours.

I'm an Anti-Braker

10 February 2015

A brilliant piece by Robert Moore:

A few weeks ago I saw a car accident - two people went through an intersection at the same time. Both slammed on their brakes at the same time and collided. Fortunately no one was seriously injured.

But then it occurred to me - if they had just gone through the intersection, they wouldn’t have collided. The brakes CAUSED the accident!

Maven incremental builds

6 February 2015

One of the issues we have at our company is that build times for our major products take a long time to complete, hindering adoption of non-incremental build IDEs such as Netbeans.

A possible way out is to adopt takari’s incremental build features. This is certainly something we are going to try out in the coming week(s).

From the Takari Lifecycle as a replacement for Maven’s default lifecycle page:

TEAM includes an optimized replacement for the Maven default lifecycle. The Takari Lifecycle Plugin provides you access to a number of significant advantages:

  1. One plugin with a small set of dependencies provides equivalent functionality to five plugins with a large set of transitive dependencies. This reduces the download times to retrieve the needed components as well as the storage space requirements in your repositories.

  2. The configuration for a number of aspects for your build is centralized to one plugin and simplified.

  3. The reduced complexity of the plugins involved in the build, results in higher build performance on the command line and in the IDE.

  4. The build is fully incremental, not only for your source code, but also for your resources, which in turn again speeds up development cycle and build times.

  5. Dedicated IDE support brings the advantages of the lifecyle to your daily development work.

The AI Revolution: The Road to Superintelligence

6 February 2015

In the category of interesting things to read when you have some time, this first part of a series by Tim Urban should be on your list:

It’s most intuitive for us to think linearly, when we should be thinking exponentially. If someone is being more clever about it, they might predict the advances of the next 30 years not by looking at the previous 30 years, but by taking the current rate of progress and judging based on that.


And everything we just mentioned is still only taking in stagnant information and processing it. To be human-level intelligent, a computer would have to understand things like the difference between subtle facial expressions, the distinction between being pleased, relieved, content, satisfied, and glad, and why Braveheart was great but The Patriot was terrible.

Read it all. Part 2 is also available.

Missing the Point of Server-Side Rendered JavaScript Apps

6 February 2015

The death of server-side rendering has been greatly exaggerated. Ember.js developer Tom Dale:

Say what you will about server-rendered apps, the performance of your server is much more predictable, and more easily upgraded, than the many, many different device configurations of your users. Server-rendering is important to ensure that users who are not on the latest-and-greatest can see your content immediately when they click a link.

Interesting things brewing in the Ember.js space.

Eclipse: confidence lost

7 October 2014

Today I lost about 2 hours of time trying to figure out why my modified code didn’t end up in my running application server, even while I use JRebel to reload classes.

It was caused by having the “Build automatically” flag turned off.

Upfront this is clearly a PEBCAK failure on my part.

What frightens me is my complete and utter distrust of Eclipse’s plugin system that made me go on a yak shaving mission in trying to find which plugin failed to work this time, instead of remembering the obvious, glaring setting that enables/disables builds.

Eclipse has failed so many times to work properly that I have lost the confidence in its ability to help me be a better developer.

And no: this is not a plight for alternative IDEs: I’ve tried them all and still think Eclipse is a better fit for me. YMMV.

iPhoto Considered Harmful

25 August 2014

Screenshot of iPhoto repairing my library

I collect my precious photos in iPhoto. iPhoto is Apple’s photo management application for families–or so I thought. Unfortunately while designing the application Apple in all its wisdom thought that the personal computer would be the ‘Digital Hub’, without taking into account that people would own multiple PC’s to access that Digital Hub–in our case our photo library.

The last few weeks I have been repairing, rebuilding and restarting my iPhoto library to no avail. iPhoto is completely and utterly unsuited as a photo library management application. It looses track of your photos, it looses metadata, EXIF information and worst of all denies you access to the precious photos of your family on a regular basis.

iPhoto’s library is not just a folder structure with your photos neatly distributed around it such as normal users would structure a photo library, but it is a structure that encompasses a database. I should give the Apple engineers credit for not putting the actual photos into said database–if you open the libary you can see the originals sitting comfortably in a folder called Masters, organized by the date of import.

To accomodate the following usecases we keep our iPhoto libraries on a NAS:

  • accessibility from multiple computers in our home network
  • redundancy in hardware through a RAID-1 setup, which uses two drives to store all data twice, also called mirrorring
  • automatic backups of our memories through autobackup tools supplied by the NAS vendor

For the last 4 weeks since I uploaded my vacation photos to my iPhoto library I cross my fingers in idle hope every time I start iPhoto that the gods that be will allow unhampered access to my photos.

I have memorized the startup key sequence necessary to coerce iPhoto to do its magic on the database:

I have started all options in vain. I had to create a new, empty iPhoto library and import all photos from my broken beyond help original library. This removed all metadata, all event information and even modified EXIF data.

Screenshot of iPhoto updating my library

With this new library in place it took two nights of work to recategorize all photos and movies into the vacations we have had over the years. We did loose all metadata associated with the photos: the detailed location information added because our 2005 camera didn’t have a GPS for example.

Insanity is repeating the same mistakes and expecting different results.

And yet when we were selecting the photos of our last vacation to put into a photoalbum, iPhoto randomly corrupted the database again. This with a clean library only 2 weeks old. I had to do the restart machine, repair permissions, rebuild library and repair database rain dance all over again. I was this close to raiding the freezer of a chicken and waving it in front of my MacBook.

I desperately await the day Apple pulls the plug on iPhoto. I hope the new Photos platform will be of this day and age and won’t force me to start using Google’s Picasa–they already read all my email and search queries, I don’t want them to own my photos too.

Can I fault the developers at Apple for this? I’d like to but I guess a multi-user design for the underlying system didn’t make the drawing board as that doesn’t demo well enough. I blame the internal system at Apple that is responsible for such design decisions for this monstrocity of an application.

After last couple of weeks’ experience I consider iPhoto harmful, and so should you.

Maven Release plugin: a lesson in pain, failure and depression

22 August 2014

When trying to release Wicket using the maven-release-plugin I’m often presented with a failed build because Maven couldn’t upload to Apache’s Nexus instance (remote closed the connection). Of course that shouldn’t happen but I’m not holding my breath for a solution anytime soon.

The failed build causes me to have to restart the release from scratch. Now folks have told me previously that I need to run

  • cd target/checkout
  • mvn -rf module deploy

instead of using the release plugin. However, this is not very pragmatic when you have to provide all parameters configured in the build script at the command line, and it goes against all my principles of having an automated build.

So I figured to configure the maven-deploy-plugin to retry uploads, and only upload at the end so I first get test failures and not a half filled staging repository when something goes awry:


Lo and behold, this doesn’t work, because the Maven Deploy Plugin no longer uploads anything during “release:perform”.

So you think: no problem I just run mvn deploy from the checkout and be done with it… Nope. Maven:

  • starts building the artefacts again and
  • testing them again (for the 3rd time) and
  • packaging them again
  • and …

How anyone can actually use this contraption effectively is beyond me. And it appears I’m not the only one.

What now?

M2E Eclipse Settings Plugin

15 July 2014

At work (Topicus) we just released a M2E plugin for Eclipse that enables you to manage your workspace settings for your team from your Maven POM file.

The M2E Settings plugin allows you to specify your Eclipse settings once, and have them delivered across your team with ease.

If you are still using the maven-eclipse-plugin to generate your project files and use its configuration jar setup you can now switch to using M2E because our plugin will maintain these settings for you.

You could configure the maven-eclipse-plugin in your project’s POM like so:


So previously you had to manually apply the settings using the maven-eclipse-plugin. This would copy all the configured files to their locations, and configure Eclipse (a restart of the workspace is typically required):

$ mvn eclipse:eclipse

And with the right configuration this would create, amongst other things for each of your projects a .settings folder with your configuration files:

$ ls -l .settings/
total 160
-rw-r--r--  1 dashorst  staff     85 Nov 29  2013 edu.umd.cs.findbugs.plugin.eclipse.prefs
-rw-r--r--  1 dashorst  staff     88 Nov 29  2013 org.eclipse.core.resources.prefs
-rw-r--r--  1 dashorst  staff    206 Nov 29  2013 org.eclipse.jdt.apt.core.prefs
-rw-r--r--  1 dashorst  staff  29892 Nov 29  2013 org.eclipse.jdt.core.prefs
-rw-r--r--  1 dashorst  staff  11754 Nov 29  2013 org.eclipse.jdt.ui.prefs
-rw-r--r--  1 dashorst  staff     86 Apr  5  2013 org.eclipse.m2e.core.prefs
-rw-r--r--  1 dashorst  staff    575 Nov  7  2013 org.eclipse.wst.common.component
-rw-r--r--  1 dashorst  staff    167 Apr 19  2013 org.eclipse.wst.common.project.facet.core.xml
-rw-r--r--  1 dashorst  staff    403 Nov 29  2013 org.eclipse.wst.validation.prefs
-rw-r--r--  1 dashorst  staff     77 Aug 25  2013 org.hibernate.eclipse.console.prefs
-rw-r--r--  1 dashorst  staff    262 Nov 29  2013 org.maven.ide.eclipse.prefs

Unfortunately with the advent of Eclipse 4.3 and its M2E integration, the maven-eclipse-plugin can no longer be used. But we still want the settings to be applied to all our projects.

In comes an Eclipse plugin that hooks up M2E and the maven-eclipse-plugin to use the same configuration and instructs M2E to fulfill the role of the maven-eclipse-plugin.

The M2E Settings plugin piggybacks on the configuration you already had in place for the maven-eclipse-plugin. This makes it easier to migrate your team members without forcing everyone to start using M2E.

For more information about the M2E Settings plugin, take a look at the project on Github and try it out. It should be added to the Eclipse Marketplace somewhere tomorrow, making it easier to find and install.

This plugin was made possible by Olivier Nouguier, so many thanks for his initial work.

Go check out the M2E Settings Plugin and have fun!

Fix wrong iPhoto upload

10 July 2014

When you select the wrong iPhoto library–say a backup copy of your iPhoto library–and upload your son’s birthday pictures, and then erase your memory card to take more pictures, you find yourself in a pile of hurt.

It is unclear if an rsync between both libraries will work: will the extra photos appear in the library or not. Will they have an event attached, etc. Your main library with over 5 years of photos of your child is not something to experiment with.

So I figured that you need to find out the files that are present in one library and not in the other, and copy them to an outside location. From that point on, you can just import the photos using the standard iPhoto tooling into the correct library.

Now my NAS (a Synology) creates additional files and folders, and OS X is also notorious for adding hidden folders. I particularly wanted to exclude those files and folders as they were not instrumental in re-importing the missing photographs.

Fortunately rsync has an option to compare two folders and synchronize the missing or updated files to a third location. The command below will copy over the missing or updated files in my master iPhoto library from my backup iPhoto library to an external folder.

Note the exclusion of the “CR2” files–I exclude the RAW versions of my photographs because I didn’t want to have to edit 250 RAW pictures. The JPGs will be good enough for this birthday party. If you care about your RAWs then remove that line.

rsync --verbose --exclude=".AppleDouble" --exclude="*Syno*" \
      --exclude="@eaDir" \
      --exclude="*.CR2"  \
      -h  \
      --archive  \
      --ignore-existing  \
      /volume2/backup/iPhoto.photolibrary/Masters/2014/ \
      --compare-dest=/volume1/Photos/Liam.photolibrary/Masters/2014 \

I can then import these photos from the /volume1/Photos/2014 folder into my master iPhoto library.

Of course note that this is provided as is and you should check if this works for your usecase before running the script on your own photo libraries. Use at your own risk!

Create your own Custom Eclipse Luna

7 July 2014

Ever dreamed of a lean mean Eclipse machine? So did we–and we think we are now living the dream!

At work we have about 30-35 developers working with Eclipse. Each developer has to download their own Eclipse bundle, and ask their co-workers which version they should get:

  • Eclipse for Java Developers
  • Eclipse for JavaEE Developers
  • Eclipse Platform SDK

Neither of these bundles are perfectly alligned with our daily needs.

We are Java developers but use Github for our code repositories. We use Maven (and have migrated our projects to utilize M2E). We have migrated from Subversion to Git a while ago. The Eclipse for Java Developers is bundled with Subversion support, Mylyn support (who uses that?) and some other plugins we don’t use. So while a good starting point it already comes with bagage.

Low-cost airlines motto for development environments: extra bagage costs more.

We are in the transition moving away from our custom monolithic platform (tomcat/spring/hibernate/wicket/cxf) on towards a more commonly accepted platform based on JavaEE 7, where we can skip direct dependencies in favor of a pre-packaged, pre-integrated container.

This migration towards JavaEE would suggest that the Eclipse for JavaEE (EJEE for short) developers would be ideal for us. But it isn’t, far from it! EJEE is slow, enables too many non-essential validations, checks and other tripe that keeps your workspace rebuilding for the next couple of months.

This means that folks would probably download the EJEE edition, and get bogged down in plugins that you need to suspend (with a low success rate), or have to go on a treasure hunt to find the right plugins.

A typical fresh and clean Eclipse installation would take about 2 hours of work to get up and running with the proper tooling and plugins installed. With Eclipse Luna now available, our company was looking for a serious productivity dip while developers are installing the latest and greatest.

Enter yoxos

In our friday “innovation day” project we discovered yoxos: a way to craft and distribute your own custom Eclipse builds and profiles. This was exactly what the doctor ordered!

We were able to craft our own custom Eclipse profile with proper initial settings (like use native polling/hooks to monitor file system changes), nice plugins (like qwickie), and without unnecessary baggage such as a JPA-validator that can’t cope with 800 entities.

All our developers can have a working custom JavaEE Eclipse setup in about 10 minutes. With yoxos we can distribute updates to our setup without any issue. And best of all: our developers can still customize their setup if they choose.

It is also possible to create custom profiles for varying projects that need a particular plugin installed (for example an Android project).

There are some (minor) issues with yoxos though:

  • the yoxos advanced launcher doesn’t work on ubuntu
  • yoxos doesn’t have a way to provision M2E connectors
  • there is no way to add your own profile to the launcher making it more easy to start your custom Eclipse
  • checking out source code for your workspace from the profile doesn’t work (as well) when using private github repositories

We’d also like to be able to provision a wildfly local server and launch configuration using yoxos, but alas that is still something we need to do manually.


Yoxos is highly recommended for crafting your own custom Eclipse distribution inside your company (or university).

Our previous setup model:

  1. download a Eclipse distribution
  2. start Eclipse
  3. wait for eclipse “rebuild workspace” to complete
  4. go to marketplace
  5. select, accept and install plugins
  6. restart
  7. wait for eclipse “rebuild workspace” to complete
  8. add update sites to eclipse to add more plugins
  9. add more plugins
  10. accept licenses and wait for downloads
  11. restart eclipse
  12. wait for eclipse “rebuild workspace” to complete
  13. remember which checkmarks you need to set to make Eclipse workable
  14. wait for eclipse “rebuild workspace” to complete
  15. try to disable a random plugin to get rid of idiotic webservice validations
  16. give up

Our new model:

  1. download Yoxos launcher
  2. start Yoxos launcher
  3. log in
  4. Download our profile
  5. Launch our eclipse profile
  6. Accept licenses and wait for downloads
  7. Wait for eclipse “rebuild workspace” to complete
  8. Start coding
  9. Profit!

Eclipse Console Wish List

3 July 2014

There are a lot of things I like about working with Eclipse. This is not a post about those things. Here I’d like to vent small frustrations I have with Eclipse that would make it feel more like a thoughtful product than something incidental.

This instance I’ll rip on Eclipse’s Run/Debug Console.

Eclipse’s debug/run console is really barebones. It’s my guess that a MS DOS 2.0 prompt gives more formatting possibilities. So here’s my wish list for Eclipse’s console:

  • default infinite scroll buffer (bugzilla)
  • ANSI escape character support (bugzilla)
  • smart scroll lock behavior (bugzilla)

There are more gripes to be had with the console, but for now let’s look at these three.

Default infinite scroll buffer

Vi, less, more, and all other utilities are capable of displaying gigabytes of log files, yet Eclipse’s console seems stuck in the Notepad era. With each new installation and workspace I have to instruct the console to use a infinite scroll buffer. Should Eclipse actually support an infinite scroll buffer, without memory overhead, why not make it the default?

ANSI escape character support

Granted most of the times I’m looking at a long list of log lines produced by SLF4J or some similar logging framework. But even in those circumstances it would be beneficial to have color coded lines (e.g. red colored lines for error level messages)

Sometimes though I have a commandline utility where I’d like to show a progress using a percentage (overwriting the previous state). ANSI control characters would allow that (and is properly supported in most terminals).

Unfortunately someone closed the feature request in Eclipse’s bugzilla instance with no option to reopen it.

Smart scroll lock behavior

Currently the scroll behavior of the console is abysmal. When you scroll back in the log to see if something went awry during startup, any logging activity will put your view back to the end of the log. You can of course lock the scroll position, but then no update will ever be visible because the position is locked. You then have to manually scroll to the end to see the action.

In my opinion scroll behavior could be quite a bit smarter when implemented as follows:

  • When I look at the end of the buffer, scroll lock should be disabled and updates should scroll before my eyes keeping the most recent update in view.

  • Whenever I scroll back, scroll lock should be enabled immediately. There is no use in moving back to the end of the buffer when something new arrives while I am looking at a stack trace.

  • Whenever I scroll forward and reach the end of the buffer, scroll lock should be disabled and updates should scroll before my eyes, keeping the most recent update in view.

I really would like to see these implemented and I think numbers 1 and 3 should not be too hard to implement.

Prevent Eclipse from stealing focus

2 July 2014

A typical workday starts like this: I open up my notebook, type in my password, fire up Eclipse using QuickSilver (⌘-space, ec⏎), fire up Tweetbot using QuickSilver (⌘-space, tw⏎), start Adium (⌘-space, ad⏎), start Mail (⌘-space, m⏎), start Safari (⌘-space, s⏎).

Any time when I try to start one such application Eclipse can swoop in with its “Select workspace” dialog, causing me to create workspaces with names “ec”, “ad”, “mail”, etc. This is because I try to get my day started quickly, and Eclipse is an asocial attention seeker stealing the input from other applications.

So one day I was fed up with the umptieth unfortunate workspace creation event and I looked up if someone had encountered this problem and thought about fixing it.

There’s a fix for this (full article): add the following two (2!) lines of XML to Eclipse’s plist file in the dist section:


This will prevent the application from stealing the focus when you are busy with other things. You can modify this yourself while waiting for the Eclipse guys to solve this issue.

Using Eclipse Mylyn for Github Pull Request Code Review

2 June 2014

At €dayjob we use github as our code repository and recently opted to require code reviews for our contributions. The benefits of code reviews require a separate post, so I won’t discuss those in detail here.

As code reviews are a given in our project, I thought that it would be awesome if I can checkout a pull request using Mylyn and have the context contain only the relevant files partaining to the pull request.

As you can imagine, a typical pull request consists of a dozen or two files, and while reviewing them in the Github web interface is doable, I miss navigating through the code in the web interface (ctrl-click anyone).

So I figured Mylyn would be a solution that would help me with my usecase. Unfortunately it doesn’t. There is apparantly no way to instruct Mylyn to checkout the branch and have all modified classes be added to the context–or nobody has thought of such a use case.

Netbeans 8 nightly: impressive first day

14 January 2014

We are busy implementing a new project that is augmented onto an existing application. We ran into some issues with Eclipse and its m2e plugin that caused us to look elsewhere–notably IntelliJ and Netbeans.

The old application is a monolithic, multi-module (51 in total) maven build that results in a couple of 140MB war files that are deployed into a tomcat server. It has dependencies on the whole non-JEE world: Spring, Hibernate, CXF to name a few. We have implemented our own cement between Spring, Hibernate and CXF, and use Spring only to bootstrap our services and perform the odd injection into Wicket applications. So while we have Spring on our class path, I would hesitate to call our application a 'Spring application'.

The new application we are crafting upon this gigantic monolith will be deployed into Wildfly. We are starting to embrace JEE as our development platform because we are tired of writing the integrations between the disparate services ourselves. Our new application consists of many Maven modules and will provide a decoupling between our database layer and our Wicket layer through a REST api. I'll discuss that in  a later post.

One of the things we needed to solve was cache synchronisation between our old application and new application: this is achieved by transforming the non-anemic entities from the old application into anemic entities for use in our new project. A custom maven plugin  strips all non-JPA properties and methods, and generates getters and setters for the properties. The plugin retains the JPA annotations.

Another thing we want to use in our new application is the JPA meta-model generation. This allows us to create typesafe JPA queries, which is great going forward in our JEE architecture.

Trying to get all this goodness to work in Eclipse forced us to upgrade Eclipse to the latest development build 4.4M4, as anything else failed to actually do anything with this monster. We were able to get this to work sort-of, but at the cost of roughly 5 man days of downloading, installing and upgrading Eclipse.

When we finally got Eclipse in a circular build, we figured that other IDEs might have a better solution to our travails. I remained sceptical since my previous encounters with Netbeans and IntelliJ have been less than successful. So one of my co-workers downloaded the latest IntelliJ Ultimate edition, and I started with Netbeans.

First I downloaded Netbeans 7.4, and tried to install the Wildfly plugin, but the module server went down right at the moment after I had selected the plugin to download. So I figured that I could just as well download the Netbeans 8 nightly since that has the Wildfly plugin already in its module index.

Long story short: I got Netbeans 8 and Wildfly integration to work with our full project setup in less than 4 hours. And everything appears to work as we expect. I am really impressed with this feat: an IDE I have never worked with for more than a couple of hours beat an IDE I have lived 10 hours a day, 5 days a week for the past 10 years with setting up this beast of a project.

Hat tip to the Netbeans community for this impressive result.

Now I'll start and actually use Netbeans in day-to-day work to see if we all should use it. I have a couple of gripes already, but I'll save that for my twitter rants, and a followup post detailing all that is wrong with Netbeans... But for now I am really happy with Netbeans and hope that it continues to impress me.

Why I don't consider IntelliJ IDEA to be the best Java IDE

4 November 2013

This is something that has bothered me for a long time. Many people I know like and love IntelliJ IDEA, but every time I try it I loathe the experience. I think the editor is stupid as it makes *me* think whether I have to override, implement or generate a method by enforcing different keybindings for those actions. I just want to press ?-space and select the appropriate action from there (using autocomplete). Eclipse gets this. Smart autocomplete in my book is to have ?-space work for all use cases: did I already type get? Then probably you want to generate a getter for a property. Did I not type anything? Then I probably want to implement (or override) a method.

Smart autocomplete in IntelliJ is anything other than smart. More often than not it presents me with an empty popup. IntelliJ wants me to press ????-space or ?????, which breaks my fingers. I have to remember which magic combination of ???? with enter or space invokes the thing I don't want to have to think about: implement an abstract method or override a super method? I already know thy name, don't make me think about the implementation detail of implementing or overriding!

IntelliJ has a user interface that makes me do work for the computer instead of making the computer do work for me: having to instruct the compiler to do its work after saving a Java file, organize imports on save, etc. I know you can record macros, but then everybody has different actions configured. Note that I typically work in a team with 5 to 15 people committing to the same code base. Having the exact same actions that are executed consistently on save is a life saver.

There is one thing that I really don't understand of IntelliJ: it doesn't make migration from Eclipse users streamlined. Yes, they provide keybindings for Eclipse users, but those only work for Windows users–not OS X users. They don't provide a working importer for Eclipse formatter rules. So when someone uses IntelliJ in an Eclipse environment the formatting is not consistent. I have used the Eclipse formatter plugin but that doesn't make working in IntelliJ more fun.

To sum up: even though I frequently throw a fit when Eclipse stops behaving correctly, I still consider it to be the best IDE for Java coding. It is unfortunate that m2e (the Maven integration into Eclipse) makes Eclipse bog down to barely useful. But that is material for another rant on another day.