Milestone 7 is the last Kepler milestone before we start rolling out release candidates — with a scheduled release on June 26th 2013.  There are a number of new and noteworthy things from the Eclipse and Equinox teams in this milestone, including:

A p2 Remediation Wizard to help you move past p2 errors. If an install fails, p2 will look for other solutions such as upgrading or removing blocking components. The results of the remediation will be presented and you can choose the path that makes sense for you.

Open Resource improvements:

and you can now use Content-Assist without a prefix

Also, the first set of Eclipse 4.x API has also been released.

If you are planning on building on, or even just using Eclipse Kepler, now would be a great time to download this milestone and help out with the testing. Less than 2 months to go!

Checkout the entire New and Noteworthy, or download milestone and try it out yourself.

http://download.eclipse.org/eclipse/downloads/drops4/S-4.3M7-201305020800/

For those of you who haven’t heard of the Raspberry Pi, it’s a low cost computer that includes HDMI (1080p) output, USB, Ethernet, an ARM processor (700Mhz) and 512Mb (or 256Mb) of RAM. The entire computer is the size of a credit card (or an Ontario Fishing License if you prefer), and can be purchased for as little as $25 ($35 for the 512Mb version). The board also includes 26 pins, allowing you to connect it to all sorts of crazy hardware — treating the little computer more like an embedded controller.

Backing up a decade, OSGi (the underlying modularity system used for Eclipse) was originally designed for embedded systems such as set-top boxes. The EclipseRT (Runtime Project) extended this work by bringing high quality runtime components to this programming model. EclipseRT includes components such as: a powerful webserver (Jetty), a provisioning platform (p2), persistence technology (EclipseLink), a rich widget toolkit for Ajax based applications (RAP), and more. The idea that you could create a full software stack based on Eclipse and deploy it on a stick has been on our minds for years, but a we lacked a low cost (and generally available) computing platform to deploy it too. With a Raspberry Pi in hand, I decided to revisit this vision.

The Setup

To get started, you need an OS and a JVM. There is a Debian based distribution for the Raspberry Pi (Raspbian) and Oracle has a JVM for the ARM processor. Unfortunately the most common setup doesn’t work. The Oracle JVM does not support the hard-float ABI used by Raspbian. Lucky for us, there is a Raspbain image that supports soft-float ABI. Download this image and write it to an SD card (there are several good tutorials out there on how to do this). Once the image is on the card, you can start your Pi and install the JVM.

 The RT Stack

The application I used was a simple web based mail client that leverages RAP (for the widgets) Jetty (for the webserver), Apache Felix Gogo shell, and Equinox. The entire application was designed as an  ’Eclipse Product’ and built with Tycho. Because Raspberry Pi is not a supported architecture, we targeted Linux/x86 (knowing full well that no native libraries would work). Once the build was completed, the product was copied to the Pi.

The Launch

The launchers that you get when you build a product are native, that is, binary executables for the particular platform. Because Raspberry Pi is not a support platform (yet), you cannot use the native launcher. Instead, you need to start your product by invoking Java and specifying the launcher jar.

 The Result

With the runtime stack and mail application running on the Raspberry Pi, testing the setup is as simple as navigating to the Pi with a web browser.

While this is just an example, it does show that you can get a non-trivial OSGi application (46 bundles) running on a Raspberry Pi without much work. It also opens the door to a large collection of Eclipse based runtimes (including the modelling technologies, ECF and more) running on the Pi. Having the EclipseRT projects running on a Pi is a great addition to the other M2M (Machine-to-Machine) work that is happening at Eclipse.org.

For more information on what I’m doing with Eclipse, Equinox, p2 and the Raspberry Pi, feel free to follow me on Twitter (@irbull).

This article describes how to install Eclipse plug-ins from an update site (p2 repository) that is hosted on an HTTP(s) server with a self-signed (SSL) certificate.

The Problem

If you try and connect to a p2 repository on a server with a self-signed cert, you will more than likely hit the following error.

Looking closely at the log, and you will see:

javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
 at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:397)
 at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128)
 at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:397)
 at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
 at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)
 at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
 at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)
 at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
 at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
 at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
 at org.eclipse.ecf.provider.filetransfer.httpclient4.HttpClientFileSystemBrowser.runRequest(HttpClientFileSystemBrowser.java:263)
 at org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$DirectoryJob.run(AbstractFileSystemBrowser.java:69)
 at org.eclipse.core.internal.jobs.Worker.run(Worker:53)

Solution

While Eclipse and p2 doesn’t offer any nice support out-of-the-box, you can easily get around this problem by installing the root certificate (or the certificate itself) into a truststore on your machine, and then use that truststore when starting Eclipse.

To install a certificate into a custom trust store, simply:

keytool -import -alias custom-ssl -file my-custom-cert.cer -keystore ~/custom.truststore

Now, when you start Eclipse, add the following arguments to your eclipse.ini file.

-Djavax.net.ssl.trustStore=<path_to_custom.truststore>
-Djavax.net.ssl.trustStorePassword=<password>

When you connect to the update site (p2 repository), the transport layer will be able to validate the self-signed certificate. If the site requires authentication you will even be prompted for a username / password:

Pro-Tip

A common question is how do you get the certificate to add to the trust store. While you could contact your system administrator and ask for the cert, it’s actually much easier than that. The certificate is public, and you can fetch it directly from your web server:

echo -n | openssl s_client -connect HOST:PORTNUMBER | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > my-custom-cert.cert

Special thanks to Gunnar Wagenknecht, David Williams and Markus Knauer for the help they gave me with this problem.

If anybody has any other suggestions for how they deal with self-signed certificates in Eclipse or Java, please feel free to leave a comment (or ping me on twitter @irbull).

While many people know Eclipse as a great IDE, did you know Eclipse also provides a powerful software stack to build enterprise applications on? From Modularity to Java Persistence to a world class Webserver, Eclipse is showing up in places you’d never expect — including the International Space Station.

While the technology is certainly world class, getting started can be daunting. Unlike the Eclipse IDE, the RT projects don’t provide an integrated solution that you can simply download and start. Are we failing the community because of this? Could we do better? Should EclipseRT projects work towards a common goal or is it just a useful organizational construct for projects that would otherwise have no home under the existing Eclipse structure? Is there a business model for this, or should that not matter? To answer these questions, we’ve assembled an RT Panel with several PMC members.

Tom Watson, Glyn Normington, Gunnar Wagenkncht, Markus Knauer, Pascal Rapicault, Doug Clarke and Mike Millinkovich will be discussing these questions on Tuesday afternoon (5:00 pm). I’ve prepared some material to get things started, however, I would really like to hear from the community. Do you have questions for the panelists? What are your thoughts on the structure of EclipseRT? Does the simultaneous release make sense or should RT be moving in a different direction? Please feel free to leave a comment here, or better yet, come to the panel and ask your questions there.

I’m looking forward to seeing everyone in Boston!

Beware the IDEs of March #eclipse

— Wayne Beaton (@waynebeaton) 15. März 2013

Included in this milestone is a migration wizard for use in shared installs. If Eclipse is installed as a shared install, and the base changes, the wizard will help you update.

There is a new JDT Quick fix for !=

And some SWT enhancements such as BIDI on buttons and customizable link colours.

Checkout the entire New and Noteworthy

http://download.eclipse.org/eclipse/downloads/drops4/S-4.3M6-201303141330/news/

or download this install and try it for yourself.

http://download.eclipse.org/eclipse/downloads/drops4/S-4.3M6-201303141330/

Also, this is the first Eclipse Milestone built with Maven Tycho (instead of PDE/Build). This entire process started over 1 year ago. Great work everyone!

• Eclipse Juno (Platform version 4.2.2)
  
• Eclipse Indigo (Platform version 3.7.2)
• Eclipse Helios (Platform version 3.6.2)
• Eclipse Galileo (Platform version 3.5.2)

You can simply download the EPP Packages as they were shipped, or you can use Yoxos to create your own custom EPP Package. You can even design your own p2 repository based off one of these releases (and we’ll host it for you). If you need even older releases, please don’t hesitate to contact us.

And now you’ll have Unicode characters in your console.

There are a number of new features in this milestone, including:

The ability to category bundles in your p2 repositories (not just features).

Null analysis for fields:

And Improvements to shared installs:

Pascal also gave some more information about the work he is doing on Shared Installs on the p2-mailing list today.

Checkout the entire New and Noteworthy, or download the Milestone and try it for yourself:

http://download.eclipse.org/eclipse/downloads/drops4/S-4.3M5-201301302000/

Scaling Up JGit
I’ve seen a number of talks about how Google structures their development infrastructure. From a single source tree holding over 2,000 projects to over 50,000 builds per day. I’m really excited to hear some of the technical details of how they manage to scale to this size. From 30Gb repositories to handling millions of Git operations per day, this talk is sure to pack a lot of stuff with very little fluff.

Marketing an Open Source Project
Many of us have plenty of experience developing software, but when it comes to talking about it, marketing it, or growing a community, we often look the other way. After all, I’d rather be hacking. This talk should help give us the tools and advice to help grow our projects and the communities around them

NASA uses Eclipse RCP applications for experiments on the International Space Station
Eclipse in space! Most people know that Eclipse is a great integrated development environment. But did you also know that Eclipse is a general purpose platform for building everything and nothing in particular? In this talk, NASA will show off an Eclipse RCP application used to control robots from the ISS. Any talk about the challenges around building software for use in space sounds pretty cool to me.

Combining the power of IBM Watson and OSGi
From Space to the most advanced question answering system in the world, Eclipse is Everywhere. I’m really excited to hear about how Equinox OSGi (the module system behind Eclipse) is being used in the Watson tooling platform. This is certain to be one of those talks where you shake your head and say “I didn’t know Eclipse could do that”.

Advanced Git: Things you need to know about Git but were afraid to ask
As more and more projects move to Git, developers need to be comfortable with the Git revision control system. Many of us simply learned Git by example and are able to use it to do our jobs. But what happens when a push fails, merge conflicts appear, or a commit goes missing? This tutorial will help prepare developers for the rare occasions when you need to dig deep into the advanced powers of Git. I’m really looking forward to learning advanced Git usage from the experts at GitHub.

Checkout the entire program on-line, or better yet, join us in Boston.

Lucky for us, the Eclipse Memory Analyzer (MAT) can help provide details of an application’s memory use. The tool is useful for both tracking memory leaks and for periodically reviewing the state of your system. In this tutorial I’ll outline 10 tips to help you use the MAT more effectively. If you’re a Java developer, the Eclipse Memory Analyzer Tool  should certainly be in your debugging toolbox.

The Memory Analyzer can be installed using the Install New Software dialog or through the Eclipse MarketPlace Client.

In this example, we are using a very simple program that allocates 100,000 Listeners, and stores them in 4 different lists. The application then sleeps without removing or clearing these lists.

1. Acquiring a memory snapshot (Heap Dump)

There are several ways you can get started with MAT. You can:

1. Configure an application to dump the contents of its memory when an out of memory error occurs,
2. Connect the MAT to an existing java process, or
3. Manually acquire a heap dump and load it into the MAT

In all cases it’s important to keep in mind that this is a snapshot of memory at a point in time. The MAT cannot tell you why an object was created, nor can it show you objects that have already been garbage collected. However, if you use the MAT alongside other debugging tools and techniques, you can often conquer memory leaks pretty quickly.

To configure your application to produce a heap dump whenever an OutOfMemory Error is thrown, add the following vm argument:

-XX:+HeapDumpOnOutOfMemoryError

Alternatively, you can use jstack to acquire a Heap dump from a currently running java process.

jmap -dump:file=heap.bin <pid>

Finally, you can use the MAT’s Acquire Heap Dump action to choose an existing Java process on your local machine.

When you load a Heap dump for the first time, the MAT will index it. This make take a few minutes, but the results will be persisted so subsequent loads will be quick.

 2. Understanding the Histogram

When you first acquire your heap dump, the MAT will show you an overview of the applications memory use.

The pie chart in the middle shows you the biggest objects by retained size. That means if we could dispose a particular instance of java.lang.Thread we would save 11.2Mb, and over 90% of the memory used in this application. While that might look interesting, java.lang.Thread is unlikely the real problem here. To get a better sense of what objects currently exist, you can use the Histogram.

The histogram shows the number of instances of a particular class and how much memory each one uses. Of course, char[], String and Object[] are unlikely the problem. To help better organize this view, you can group by classloader or package. This will allow you to focus on your Objects.

The histogram can also be filtered using a regular expression. For example, we can show only classes that match the pattern com.example.mat.*

With this view, we can now start to see that 100,000 Listener Objects are alive in the system. We can also see the amount of memory each Object is using. There are two calculations, Shallow Heap and Retained Heap.  Shallow heap is the amount of memory consumed by one object. An Object requires 32 (or 64 bits, depending on the architecture) for each reference. Primitives such as integers and longs require 4 or 8 bytes, etc… While this can be interesting, the more useful metric is the Retained Heap.

3. Understanding the Retained Heap

The retained heap shows the sum of the shallow heap size of all objects that would be removed when this object is garbage collected. For example, if an ArrayList held 100,000 items, and each item required 16 bytes, then removing the ArrayList would free 16 x 100,000 + X, where X is the shallow size of the ArrayList. (Note: this assumes that these objects are only being referenced by the ArrayList, and not elsewhere).

The retained heap is computed by adding the size all the objects in the retained set. A retained set of X is the set of objects which would be removed by the GC when X is collected.

The retained heap can be calculated in two different ways, using the quick approximation or the precise retained size.

By calculating the Retained Heap we can now see that com.example.mat.Controller is holding the majority of the memory, even though it’s only 24 bytes itself. By finding a way to free up the Controller, we can certainly get our memory problem under control.

4. The Dominator Tree

The key to understanding your retained heap, is looking at the dominator tree. The dominator tree is a tree produced by the complex object graph in your system. The dominator tree allows you to identify the largest memory graphs. An Object X is said to dominate an Object Y if every path from the Root to Y must pass through X. Looking at the dominator tree for our example, we can start to see where the bulk of our memory is leaking.

By looking at the dominator tree, we can easily see that it’s not the java.lang.Thread that’s the problem, but rather the Controller and the Allocator that hold the memory. All 100,000 Listeners are being retained by the Controller. By either removing freeing these objects, or freeing the lists that they contain, we can likely improve our situation. There are a few useful properties of the dominator tree:

• All Objects that belongs to a subtree of X (com.example.mat.Controller in this case) are said to be in the retained set of X
  
• If X is the immediate dominator of Y (Controller is the immediate dominator of the Allocator), then the immediate dominator of X (which is java.lang.Thread in our example) also dominates Y.
• The parent child relationship in the tree do not necessarily correspond to the relationships in the Object graph

From the Histogram you can also choose a particular class and find all the objects that dominate the instances of this class.

5. Exploring Paths to the GC Roots

Sometimes you have a large collection of Objects that you’re certain your disposing of. Finding the the dominator may help, but often you want the exact path from that Object to the roots. For example, if I now properly dispose of my Controller, then certainly my memory problem should go away,  unfortunately they didn’t.  If I now choose an instance of the Listener, and look at Paths To GC Roots, I can see that the Controller class (Note: the class, not an Object) reference to a list of Listeners. This is because one of the list was declared static.

You can also explore all the incoming and outgoing references to an Object. This is very useful if you want to see all the references to a particular Object in the Object graph.

6. The ‘Inspector’

The Inspector provides detailed information about the currently selected Class or Object. In this example we can see that the currently selected ArrayList contains 100,000 elements and references an object array at memory location 0x7f354ea68.

Keeping the Inspector and the Snapshot linked will provide you with important statistics about all your selections.

7. Common Memory Anti-Patterns

The MAT provides reports for common memory use anti-patterns. These can be used to get an idea of where memory leaks are occurring, or by looking for some low hanging fruit which can be cleaned up to help improve performance.

The Heap Dump Overview will show you detailed information about your Heap Dump and provide links to common tools (like the Histogram). Information such as the threads which were running, total number of Objects in the system, size of the heap, and are also shown.

The Leak Suspects report displays possible memory leaks, and provides links to the tools and charts to analyze these findings.

Another common anti-pattern is the use of a large number of collections, with very few entries in each one. For example, if our listeners each had an array of notifiers (items that need to be notified of certain events), but these notifiers were only used occasionally, we would end up wasting a lot of space. The Java Collections tools can help with these problems.

By running a Collection -> Fill Ratio Report we can see that there are 100,000 ArrayLists that are empty. If were allocated these in a lazy manner (only when needed), we would save almost 8Mb.

We can also use Collection Analysis to see array fill ratios, collection size statistics and map collision ratios.

8. Java Tools

The MAT has a number of built in tools to generate reports tailored to the details of the Java runtime. For example, the Threads and Stacks report will show details about all the treads in the system. You can see the local variables which are currently kept alive on each stack.

You can Find all the Strings in the system that match a particular pattern:

Or even find Strings in the system which contain wasted space in their character arrays (often due to repeated use of substring).

9. Object Query Language

As we’ve shown, the Eclipse Memory Analyzer has a lot of tools to help track both memory leaks and excessive memory use. While most memory problems can likely be addressed using the techniques described above, a Heap Dump contains much more information. The Object Query Language  (OQL) allows you to build your own reports based on the results of a Heap Dump.

The OQL is an SQL-like language. Just think of Classes as tables, Objects as rows and Fields as columns. For example. to show all Objects of type com.example.mat.Listener, you would simply write:

select * from com.example.mat.Listener

Columns can be configured using different fields, such as:

SELECT toString(l.message), l.message.count FROM com.example.mat.Listener l

And finally, the WHERE clause can be used to specify particular criteria, such as all the Strings in the system which are not of the format “message:.*”

SELECT toString(s), s.count FROM java.lang.String s WHERE (toString(s) NOT LIKE "message.*")

10. Exporting your results

The memory analyzer is a great tool for creating reports about the state of an application’s memory. A Heap Dump contains valuable information about the state of your system, and the MAT provides the tools needed to access this data. However, like with many open tools, if something is missing you’re not locked in, nor are you out of luck. With the MAT you can export your results to several different formats including HTML, CSV or even Plain Text. You can then use your favorite spreadsheet program (or your own tool) to continue your analysis.

The Eclipse Memory Analyzer is a powerful tool, one all Java Developers should be familiar with. Tracking memory leaks and other memory related problems is often challenging, but hopefully with the MAT you can get to the root of your problems relatively quickly.