Archive for the ‘Software Development’ Category

Questioning Dogma

I am an increasingly skeptical developer. Things that make me think twice…

Shouting
  1. Whenever someone whips out the “premature optimization is the root of all evil” quote. I think most programmers do not realize this quote is taken out of context. The quote refers to “micro-optimization”, not large-scale architectural decisions. Many performance-related decisions DO have to be made early in a project. Thinking about performance is not the same thing as obsessing over micro-optimization.
  2. On a related note, many people (in classes I teach) are resolute in their belief that ORM tools like Hibernate or JPA are simply too slow. Sigh. These people are obsessing over performance too much, as opposed to ignoring performance considerations.
  3. Striving for 100% code coverage when testing. I’ve discussed this before, and subsequent comments and discussions have led me to believe that 100% code coverage is not all that important. It is probably more important to demand better tools that ignore trivial code, to ensure we do test things that really need testing. Filtering out “trivial” code and alerting us to complex code that probably SHOULD be tested…now that’s an interesting problem!
  4. Whenever people argue for or against Ruby/Rails, I am skeptical. Sorry, but the claims (on both sides) are just too nutty and people are too passionate to be objective. It’s not that black and white. Let’s just sit back and watch the comments this provokes…
  5. I feel the same way about Linux, Mac, and Windows. These are such heated topics that I tend to be skeptical of everyone. In fact, too much fan-boy-ism is a repellent for me.
  6. People who preach against Big Design Up Front (BDUF). Be careful, for many of these people warp the theory into “no design is ever good”. I’m no fan of BDUF, but I certainly think about many aspects of design very early on! For example, if I know my app needs to store billions of records, you can be damn sure I’ll be thinking about storage options and sorting algorithms from a completely different angle than for an app storing 300 records.
  7. The notion that unit tests are THE documentation. Uh, no. Tests are for testing. They have a happy side effect of serving as good examples, and I often refer to tests to learn how to exercise an API. But to argue that the whole point of writing tests is to create documentation…that seems wrong to me.
  8. Speaking of testing, people who rail AGAINST testing make me wonder…has this person EVER written a test? Maybe they make good points, but to NEVER write tests borders on incompetence.
  9. XML is all evil, all the time. Wrong.
  10. Ant sucks! Maven rules! Wrong. Or…Maven sucks! Ant rules! Again, wrong.
  11. YAGNI. To a certain extent, but you still need to anticipate and plan for the future.
  12. We need to lock down PCs. Restrict Internet access. Force procedures. Ick. Those things make me unhappy and I am very skeptical these techniques work. In fact, I believe as you clamp down on people, the best and brightest will leave.
  13. SOAP sucks! Web services suck! Yes, I agree. (what did you think I’d say???)

What are you skeptical about? Hopefully someone says no-name bloggers with over inflated egos… :-)

Legacy-Proof Your Frameworks

You know the drill. You design a perfect framework, release it to a group of programmers, and receive rave reviews. Yeah! Then you start version 2, but discover all those happy customers are actually using your API in ways you never intended…or imagined.

Public API

Sure, it’d be nice to rename that method…but you cannot, because that will break someone else’s app. Every time you change something, other programmers have to update their own code, which costs money and time. Features quickly become frozen and virtually impossible to change. You now have legacy code. What happened?

Minimize and Segregate

This article examines two goals when sharing code. First, you should make the public API as small as possible. The fewer things you share, the less tightly coupled clients are. If you share a public API with 500 members, clients have 500 potential connection points to your framework. An API with 20 members has far fewer couplings with client code.

Second, you should segregate public code from “private” code. Make it very clear which parts of your framework clients are supposed to use. In some cases, you can introduce barriers that make it harder for clients to accidentally use internal implementation details.

Hopefully you can guarantee stability with a very small, well-segregated public API, and retain some freedom to improve hidden implementation details behind the scenes.

Minimize Visibility

We’ll start with an easy one. Rather than blindly writing public getters/setters for every field, only expose what you really need. Use the private keyword liberally; only use public for things that need to be available to callers.

Basically, minimize visibility whenever possible. This helps keep your public API small.

Hide Behind Interfaces

Interfaces provide another mechanism to hide internal implementation details. Programmers commonly define interfaces with accessors:

public interface Name {
  String getFirst();
  String getLast();
}

As you can see, the Name is immutable. We don’t see constructors in interfaces. Interface-heavy APIs play nicely with dependency injection tools like Guice or Spring and can significantly reduce the need to code directly against implementation classes.

Use an impl Package

Interfaces need implementations. Where do these go? And what about “helper classes” like LangUtils or ComparisonHelper?

I recommend putting implementation details into an explicitly-named impl package. This gives a clear cue that everything in that package is an implementation detail that may change in future releases.

Keep in mind that Java packages only provide a hint to programmers. It is very likely that many classes in your impl package need to be public in order for other parts of your framework to use them.

Provide Static Factories

So how do you encourage people to only use public interfaces but avoid classes in the impl package? You could encourage them to use dependency injection. Or, you can borrow a technique from the Glazed Lists project. Check out the GlazedLists class.

This is a class in the “public” package with static factory methods that return interfaces. Internally, however, these factory methods construct implementation-specific classes from non-public packages.

Although this does not guarantee programmers avoid implementation details, it does provide an officially-supported mechanism to get to the implementations.

Use Exotic Class Names

Taking the impl package one step further, you might want to name implementation-specific classes using awkward names like _MyApiImpl_LangUtils. It is quite easy for programmers to inadvertently include an impl package in an import statement.

But seeing a butt-ugly class name like _MyApiImpl_LangUtils directly in the source code makes it significantly more obvious that programmers are using something they should avoid.

Include Weird Methods in Interfaces

While it is usually good to encourage interfaces, sometimes you might prefer people extend an abstract base class instead. As soon as you add a method to an interface, everybody implementing that interface is broken.

However if people extend a base class, you can add new methods to that base class without (usually) affecting people.

For Example…

Check out the Matcher code from the Hamcrest project:

public interface Matcher<T> extends SelfDescribing {
  boolean matches(Object item);

  /**
   * This method simply acts a friendly reminder not to implement
   * Matcher directly and instead extend BaseMatcher. It's easy to
   * ignore JavaDoc, but a bit harder to ignore compile errors .
   *
   * @see Matcher for reasons why.
   */
  void _dont_implement_Matcher___instead_extend_BaseMatcher_();
}

Another Reason…

I just used this technique for another reason. I’m writing a little P2P framework and each node in the network has an Address, which is an interface. Behind the scenes, however, the framework has to downcast to an implementation-specific subclass of Address. Without getting into too much gory detail, adding a huge ugly method to the interface helps remind programmers to get their Address from my framework, rather than by implementing the interface directly.

Require a Key

This is a little hack I’m quite fond of. Suppose you have a public factory:

public class MyFactory {
  // public class, but private constructor
  public final class Key {
    private Key() {}
  }

  public static Widget createWidget() {
    return new WidgetImpl(new Key());
  }
}

Let’s study that for a bit. The Key has a private constructor, so MyFactory is the only class that can construct it. This forces programmers (unless they use reflection tricks) to go through our createWidget() method to construct new widgets.

Here is how you’d write the WidgetImpl class, ideally in an impl package:

public class WidgetImpl implements Widget {
  public WidgetImpl(MyFactory.Key key) {
    if (key == null) {
      throw new IllegalArgumentException("Please use MyFactory");
    }
  }
}

Since you cannot construct a Key (again, without reflection tricks), you must use the factory to construct new Widget objects.

(my apologies if the above has a typo or something…I’m coding in WordPress…)

Not the JarJar from Star Wars…

You might consider a tool like Jar Jar Links to hide the fact that your framework uses some specific third party JAR file.

Summary

I hope you find some of these techniques useful. Some people will always try to bypass your public API and rely on implementation details. Who among us has NEVER used some com.sun.* code in our own apps? Ahem…moving on…

The point is that framework developers need to make it very clear which interfaces, classes, and packages are considered “stable” and part of the “public” API. Good frameworks will make every effort to maintain stability of these public APIs from release to release.

Although you could rely on JavaDoc comments or simple package names, it is better to make it as hard as possible for programmers to accidentally rely on internal implementation details. Code completion in modern IDEs makes it really easy to accidentally import something from an implementation package.

Just Fixed a Threading Bug…

…I think.

Now I wonder…did I really fix this thing? Or perhaps the problem occurs once per month instead of several times daily?

Just Say No

I really enjoyed reading Ask 37signals: Installable software?, in particular I enjoyed reading the comments.

(I think) many of the commenters just don’t get it. I’m talking about the people who said things like…”yeah, but what if you supported feature X? My company really wants that feature! Or what about features Y, Z, and Q? Then you could get a whole bunch of new customers!”

Learn to Say No

I recall reading an article about Google’s constant internal battle to keep crap OFF the home page. Every department wants “just one more link…”. The problem is, catering to this sort of request — trying to please everybody — results in slop that delights nobody.

I think 37Signals is right. Their software is not right for everyone, but if it were, it wouldn’t be great for anyone.

Usability Blunder: Really, Really Remember Me?

Watching non-experts perform tasks often exposes hidden weaknesses in software. Experts are often blind to obvious usability blunders. Yet watching someone else fumble through our beloved software makes the flaws immediately evident.

Case in point. A few weeks ago I was sitting with a non-geek as she navigated a web site. When prompted for a username/password, she dutifully clicked the “Remember Me” checkbox:

Remember Me

Her reward for clicking the checkbox? A completely redundant prompt from the browser:

Confirm Remember Me

To which she blurted out…something like…

WHAT’S WRONG WITH THIS !!#*($! THING??? I JUST TOLD IT TO REMEMBER ME!

And I thought…that’s such a piece of crap, yet I never even noticed it until now. In my mind, I recognize the checkbox as a feature of the web site, while the dialog box is a completely separate application - the browser.

Yet to a normal person, it’s all one thing. In this case, a redundant, stupid, annoying user interface. It’s no wonder people struggle with computers.

Final Thought

Do usability testing. Nothing fancy, just sit back and watch people use your software. Don’t tell them what to click on or how to do things. Instead, ask them to do certain tasks.

You will be surprised, I guarantee it.

GWT First Thoughts

With a sick child at home, I had some time this afternoon to play with Google Web Toolkit (GWT). Learning GWT has been on my “technologies to investigate” list for far too long; today I finally dove in. First impressions:

  • GWT is quite a bit simpler than I thought it would be. I guess all that procrastination was for nothing.
  • Having two monitors really helps the learning process. There are a huge number of getting started articles and tutorials, so having a web browser open on one display to read examples while coding in an IDE on another display is almost essential.
  • Just dive in and start coding. I’ve attended GWT presentations, watched online videos, read (some) docs…but nothing compares to writing code.
  • One of the first things I created was a TabPanel. I was someone surprised by the complete lack of a default style. Until you start messing with CSS, it looks nothing like a tabbed pane. I got completely sidetracked hacking CSS, which took vastly more time than writing the Java code to create the tabbed pane in the first place! :-)
  • It seems like there is an opportunity for talented CSS designers to put together GWT Theme Packs, consisting of ready-to-use CSS and images that give your GWT app a particular look.
  • The biggest surprise (for me) was the fact that GWT still uses Java 1.4 syntax. I guess I just assumed it would be Java 5 based on experience with other Google Java APIs.
  • Other than the CSS, writing a GWT app feels nothing like writing a web app. (this is a very, very good thing…)
  • They’ve done a great job providing a usable development environment out of the box. Far too many other tools leave IDE and app server configuration completely up to the developer. Thus, when starting with most technologies, you spend a great deal of time tracking down JAR files, configuring Tomcat, coming up with a custom development “flow” to compile, deploy, bounce Tomcat, etc. GWT comes with tools to setup the initial project, hot deploy, and test.
  • A bit more on the previous bullet…they’ve also managed to do that without locking you into a particular IDE like Eclipse or NetBeans. I find it disturbing when vendors try to lock me in to a particular IDE by writing tutorials and plugins that assume I only use one specific IDE and app server combination. So far, GWT works great in IDEA. I’m sure I could easily switch to Eclipse and everything would work there as well.
  • I plan to continue this journey.

No More Crash, Boom!

Last week, I wrote about “too many open files” exceptions we saw in our build agents shortly following large Subversion checkouts with the wonderful SVNKit library. At the time, I could not confirm if it was my code or SVNKit causing the issue.

After patching all of my code as described in the earlier post, I checked the SVNKit 1.1.x branch, and discovered they had applied the exact same fix to their own code just a few weeks ago.

Awesome Service

I contacted the people at SVNKit and heard back after a day. They let me know that the 1.1.x branch is continuously tested and is considered stable enough for use, so I went ahead and installed in.

I am happy to report our problem is now gone, the number of open files is consistently low. Before this fix, numbers would shoot up in to the hundreds, and would eventually cause the dreaded “Too many open files” exception.

+1 for SVNKit!

Technology Foothold 101

I do a lot of new technology investigation and I can tell you that first impressions matter.
Far too often, my first few hours with a product consist of:

Rock Climber
  1. Trying to figure out what to download
  2. Searching for simple examples
  3. Fighting CLASSPATH and JAR file hell
  4. Dealing with obscure “Name not found” types of exceptions

If my first day with a new product is miserable, I’ll probably move on to your competition.

Foothold 101

Here are some tips to make your product easier to approach and learn:

  1. Ease the JAR file pain.
    • Reduce the number of JARs, if possible. Apache ActiveMQ lets you choose between a single big JAR or many small JARs. Having that single JAR is a blessing during early development.
    • Clearly list the requirements for each JAR. If your product ships with 30 JARs, provide a documentation page that tells us which ones we need, and why.
  2. Provide 5-minute tutorials with simple example code. Many projects on The Codehaus have these 5-minute tutorials, and they are a fantastic idea. You can always follow up with more in-depth documentation, but the quick-start Hello World is — and always will be — essential.
  3. Beware overly technical documentation. It is highly likely that new users are not experts in arcane P2P networking terminology. The JXTA project documentation is a really good example of what not to do. I struggled with this for an entire day and just gave up in utter frustration and rage, unable to get much of anything to work.
  4. Your web site needs to be clean and easy to navigate. I am generally lost and confused when I try to navigate java.net projects like SwingLabs. Hell, their own search engine just crashed when I tried searching for “swingx”. I found it (and SwingLabs) through Google.
  5. Make sure your damn demos work.
  6. If your project requires me to compile, then let me type a single command to make it happen. I don’t care if it is Maven or Ant, just make it work. I’m here to learn about your API, not to diagnose a sloppy build.
  7. Framework and API developers should spend a huge amount of effort providing useful error messages. There’s nothing worse than firing up a new product and seeing messages like:
    • Logging is not configured. To see output edit some magical XML file.
    • Name not bound. (Uh…where did you look?)
    • Batch update failed. (gee, I don’t suppose you could tell me a table or column name?)
  8. Provide obvious downloads. If a particular release consists of three ZIP file choices, explain what those choices mean. Which do I want, and why? I generally just download all of them, because I’m afraid I’ll miss something. Check out the download counts for Guice. Notice that pretty much everybody who downloads guice-1.0-src.zip also downloads guice-1.0.zip. I’m not alone! Maybe all but the biggest projects should offer a single ZIP to download?
  9. Make sure your ZIP file extracts to a subdirectory. Don’t you hate it when you unzip some new toy, only to find it just dumped everything into the current directory?
  10. Provide an IDE-friendly ZIP or JAR containing all source code. This way I get proper JavaDoc tooltips and can easily navigate to source. Few things are more annoying that projects that ship with 15 separate parallel source trees, requiring me to spend precious time configuring a complex project.
  11. Screencasts are a great way to get people started.
  12. Concise documentation is often better than verbose documentation. Up-to-date documentation is priceless.

Credit

For the record, I stole the term “Foothold” from Mike.

Static HTML Will Do

I just ran across a really good web site consisting of what appears to be nothing more than static HTML, updated about once per month. As far as I can tell, there is nothing interactive about the site, such as the ability to submit comments. On an About page, they list the following technologies:

Less
  • JSP 1.2
  • JSTL 1.0
  • Java Servlets 2.3
  • Struts 1.1
  • JDBC 3.0
  • MySQL 4.0.13
  • Tomcat 5.0
  • Spring 1.2
  • Hibernate 2.1

Holy balls. I wonder how well that holds up to the Slashdot Digg effect compared to a site based on static files?

More

Tech Overkill

My hunch is that this is just someone’s passion and they used these tools because…well…they wanted to. That’s fine for a personal web site — do what makes you happy.

But I wonder how many “real” apps go to such technology extremes when something as simple as static HTML with perhaps some custom publishing scripts are sufficient, if not better?

The Problem with Emma…

Emma is a code coverage tool with great potential. I (sort of) like it and use it (well, until I got sick and tired of seeing huge numbers of “uncovered” sections that were, in fact, IMPOSSIBLE for me to cover). This comment from Tony makes a great point:

One of the ways that 100% code coverage really helps a project is you quickly pick out those classes that are not tested fully. The difference between 87% coverage and 88% coverage doesn’t show up as much of a difference.

But, the difference betweeen 99% and 100% shows up as a huge difference when you use tools like Clover.

This sounds an awful lot like unit testing. When 100% of your tests normally pass, you immediately recognize failures.

If your team habitually tolerates less than 100% test success, you may will fail to recognize new test failures. Test failures are lost in the noise. Over time, developers learn to expect failing tests, ignore the results, and stop running the test suite.

The best way to avoid test rot? Ensure your build FAILS if a unit test fails.

The Problem with Emma

Achieving 100% test success is easy if you start on day 1 and never let failures work their way into the code. Although this sounds similar to code coverage, it is not.

In fast, it is practically impossible to achieve 100% coverage with Emma. Some branches are impossible to reach, such as private constructors. The same is true for the default case in some switch statements, or some of the generated code in enums.

Thus, when you use Emma, you never get a “green bar”. Instead, you constantly have to wade through false “not covered” indicators to determine if you actually forgot to cover something important.

What Can Be Done?

I believe Emma is off to a great start, but it really needs to be smarter about data flow analysis. Private constructors come to mind first: they should not be reported as “uncovered”.

Secondly, I think it would be really nice we had a database of previous runs. It would be great if we could review the coverage report, and mark sections of code as “I know this is not covered, please stop warning me about this.”

If this tool were really smart, it would recognize when the code changes, and clear these markers for those sections.