+1 (669) 231-3838 or +1 (800) 930-5144

Top 10 Things For New OpenDaylight Developers

I recently started contributing upstream to the Open Daylight project (ODL) as a developer. I mostly followed the ODL developer documentation on how to get started and how to use the tools. Through ambiguous documentation or through hubris (every developer always thinks they know what they are doing), I made some mistakes and had to learn some things the hard way. This article is an enumeration of some of those hard knocks, just in case it can help the next person following this path.

First, this list is more than ten items. It is just that “top 10” has a catchy sound to it that just isn’t there for “top 17”.

Gerrit Workflow

Gerrit has a different workflow that you are likely using in downstream coding with other tools. There is a general tutorial and an ODL coding introduction to working with Gerrit that is very helpful.

Coding Conventions

We followed our own coding guidelines and those did not match to the Google coding conventions used upstream. Differences we ran into were:

  • Lower-case method names, even for JUnit test methods
  • Using the most restrictive visibility for access qualifiers (I won’t push back the philosophy of library design)
  • Use Java 8 lambdas everywhere possible

Gerrit Commands

Gerrit (plugin) has some non-obvious controls, namely that you can kick off another build by putting “recheck” as your comment. Others are “rerun”, “verify” as in here.


Upstream coders usually add the prefix “WIP: ” to their bug report to let other developers know things are not ready for prime time reviews yet. I have been putting the “WIP:” prefix as a new comment right after my new patch set.


Mechanically you can review the issues by using “Diff against:” drop list to pick a revision to start at then go into the first file and then use the upper right arrows to move between files.

Broken Master

The master branch on ODL can and does break, causing lots of down time with broken builds (days even). Full builds take 10 hours and verifying-before-pushing is an evolving process. Have patience.

Git Review

If you are using “git review” and you forgot the “- – amend” option on your commit, you will be prompted to squash and will end up with a new branch. You can recover using “git rebase -i HEAD~2” then using the force option and then abandoning the new branch.


Along with master breaking, master can also produce stale artifacts, so don’t assume that clearing your m2 cache, creating a subproject repo and building will give you up-to-date code.

Searching Jars

You can search jar files using find . -type f -name ‘*.jar’ -print0 |  xargs -0 -I ‘{}’ sh -c ‘jar tf {} | grep “YourClassHere” &&  echo {}’ for helping verify your m2 has the latest.

Patch Workflow

The patch workflow is not very good at all for having multiple people working on an item; traditional git branches and management are superior in this respect so expect some pain.

Broken Master Again

If your patch won’t build because master is broken, you can rebase it to pick up a fix temporarily. When you do a “git review” to push it, you will be prompted because there are two commits (your new patch revision and the fix). You do not want to merge that fix into your patch.

Skip Tests

There may be a bug here or there in dependencies, so you should always to a full build within a subproject the first time after you create the repo. In my case, I was in netconf and saw a difference between starting off with a “-DskipTests” or not. The former lead to a failed build, while starting with a full build (and then doing any number of skip tests), seemed to work.


If you are a developer who works with meaningful coding standards, you will find yourself clashing with the pedantic nature of ODL’s use of check style. Although it probably varies from project to project, your code reviewer might decide that your code is the perfect opportunity to enforce check style.

Bug Report

Put the URL to the gerrit issue in “external references” on the bug report and put the bug as the first part of the commit message like “BUG-8118: ….”

Gerrit Patch Set

Any replies that you make to the patch set discussion use the current patch set as an annotation. Be sure to move your “current patch set” by using the upper right drop list “Patch sets 8/8” (for example).

Easy Debug

You can do development on a project like netconf and start a controller with your code: git clone, mvn clean install, go to netconf/karaf/target/assembly/bin, do ./karaf, at console install your feature, like feature:install odl-netconf-callhome-ssh (and maybe odl-mdsal-apidocs). Viola!


If you work on the docs, then you need to know there are three kinds: contributor (on wiki), developer, and user (both on docs.opendaylight.org).

Getting Started Upstream in ODL

By Allan Clarke and Anil Vishnoi



Getting started as an upstream contributor to OpenDaylight is not easy. The controller ecosystem is big, there are many projects, and there are millions of lines of code. What is a new ODL developer to do? Here is some pragmatic advice on where to begin to become an active contributor.

Fix Bugs

One of the easiest ways to get to know a code base is to start fixing bugs. Peruse the ODL bugs list on Bugzilla, say with the NETCONF project. You want to find bugs that aren’t likely being worked on and are of limited scope (to match your limited understanding of the project). Ideally bugs will have an owner assigned to indicate that they are actively being worked on, but it is not always a great indicator. In particular, someone may run across a bug, file a report, then jump into fixing it—and forget to assign it to themselves. This is most likely with the project contributors, so figure out who are the project contributors and look at the date of the report. If it was a project contributor and a newish date, then that bug might be being worked on. You should read through the report and try to decide how much domain knowledge is needed—as a newbie, smaller is better.

Once you have selected a bug to work on, click on the “take” link. Also add a comment to the bug. If someone already is working on it, they should get a notice and respond. You can also try the ODL mailing lists and give notice there. You mainly want to avoid duplicate work, of course.

Review Patches

Reviewing patches is a great way to contribute. You can access patches via Gerrit, and we’ll use the NETCONF patches as an example. Doing code reviews is a great way to not only see existing code but also to interact with other developers.

  • If you have some domain expertise and know the code, you can review the functionality that is being pushed.
  • If you have neither of these, you can do the review based on Java best practices and good software engineering practice.

Address Technical Debt

ODL uses Sonar for analytics of the upstream project. Here is an example for the NETCONF issues. Note that the ODL project has coding conventions, and the Sonar Qube has some best practices. This list shows violations that should be addressed. As a newbie, you can work on these with little domain knowledge required. You can also see that the code coverage varies for the NETCONF coverage, so adding NETCONF unit tests to boost the coverage in the weakest areas would be very helpful.

Sonar has a lot of interesting metrics. You can explore some of them starting here including coverage, tech debt, etc. If you look at the Sonar dashboard, it will point out a lot of available work that does not require a large span of time to invest. Doing some of this work is a great step towards getting your first patch submitted.

Follow Best Practices

With well over a million lines of code and many contributors from many companies, the ODL project has quite a girth. To manage the code entropy, ODL has some best practices that you should become familiar with. These cover a diverse set of topics, including coding practices, debugging, project setup and workflow. We strongly recommend that you carefully read these. They will save you a lot of time and will pay back your investment quickly. They will help you skate through code reviews. These practices are really time-tested advice from all the ODL developers, so don’t ignore them.

Support Attribution

Attribution is an important insight into most if not all open source projects. Attribution allows stakeholders to see who is contributing what, from the individual up through sponsoring companies. It allows both a historical and current view of the project. You can see an example of why attribution is illuminating here. You need to sign up for an ODL account, and a part of that process will be to associate yourself with a company (if applicable). You can also see breakdowns by authors on the ODL Spectrometer.

That’s all for now. Happy trails, newbie.

Watch for Allan’s blog next week where he will share his Top 10 learnings as a new developer contributing to ODL.

Service Providers Are Placing Big Bets on Open Source Software Networking – Should You?

The service provider market is undergoing earth-shaking changes. These changes impact the way that applications for consumers and business users are deployed in the network and cloud as well as the way that the underlying data transport networks are built.

At Lumina, we’ve had the chance to work with several large service providers on their software transformation initiatives and get an up-close look at what works and what doesn’t. Three factors are particularly favorable in setting up successful projects for frictionless progress from the lab through prototype and proof of concept and into the production network.

Top-Down Advantage

Our first observation is that top-down initiatives and leadership work better than bottom-up or “grass roots” approaches. The experience of AT&T strongly illustrates the advantage. While a few of the hyperscale cloud providers had already launched open source initiatives and projects, the first big move among the established service providers was AT&T’s Domain 2.0, led by John Donovan in 2013. Domain 2.0 was not a precise description of everything that AT&T wanted to do, but through that initiative, the leadership created an environment where transformative projects are embraced and resistance to change is pushed aside.

While lack of top down support is not a showstopper, it is extremely helpful to get past obstacles and overcome organizational resistance to change. If top-down support in your organization is lacking or weak, it is worth your effort to influence and educate your executives. In engaging executives focus on the business value of open software networking. The advantages of open source in software networks include eliminating lock-in and spurring innovation. As our CEO, Andrew Coward, wrote in his recent blog, Why Lumina Networks? Why Now?: “Those who build their own solutions—using off-the-shelf components married to unique in-house developed functionality—build-in the agility and options for difference that are necessary to stay ahead.”

Although it may create a slower start, from what we have seen, taking the time to do early PoCs to onboard executive support so that they deeply attach to the value is time well worth spent. Sometimes a slow start is just what is needed to move fast.

Collaboration through Open Source

The second observation is that industry collaboration can work. I read an interesting comment by Radhika Venkatraman, senior vice president and CIO of network and technology at Verizon, in her interview with SDxCentral. She said, “We are trying to learn from the Facebooks and Googles about how they did this.” One of the best ways to collaborate with other thought leaders in the industry is to join forces within the developer community at open source projects. The Linux Foundation’s OpenDaylight Project includes strong participation from both the vendor community and global service providers including AT&T, Alibaba Group, Baidu, China Mobile, Comcast and Tencent. Tencent, for one, has over 500 million subscribers that utilize their OpenDaylight infrastructure, and they are contributing back to the community as are many others.

A great recent example of industry collaboration is the newly announced ONAP (Open Network Automation Platform) project. Here, the origins of the project have roots in work done by AT&T, China Mobile and others. And now, we have a thriving open source developer community consisting of engineers and innovators who may not have necessarily collaborated in the past.

These participants see benefits of collaboration not only to accelerate innovation but also to build the software run time across many different types of environments and use cases so as to increase reliability. Providers recognize that in their transformation to software networks there’s much they can do together to drive technology, while using how they define and deliver services to stand out from each other in the experiences created for customers.

What about your organization? Do you engage in the OpenDaylight community? Have you explored how ONAP can help you? Do you use OpenStack in your production network? And importantly, do you engage in the discussions and share back what you learn and develop?

Pursuit of Innovation

A third observation is the growing ability for service providers to create and innovate at levels not seen before. A prime example here is the work done by CenturyLink to develop Central Office Re-architected as a Datacenter platform to deliver DSL services running on OpenDaylight. CenturyLink used internal software development teams along with open source and Agile approaches to create and deploy CORD as part of a broad software transformation initiative.

One might have thought that you would only see this level of innovation at Google, Facebook or AWS, but at Lumina we are seeing this as an industry-wide trend. The customer base, business model, and operations of service providers vary widely from one to another based on their historical strengths and legacy investment. All have an opportunity to innovate in a way that advances their particular differences and competitive advantages.

Closing Thoughts

So we encourage you to get on the bandwagon! Don’t stand on the sidelines. A combination of leadership, collaboration and innovation are the ingredients you need to help your organization drive the software transformation needed to stay competitive. There is no other choice.

Stay tuned for my next blog where we will discuss some of the specifics of the advantages, development and innovation using open source.

Pin It on Pinterest