Tuesday, August 26, 2008

Entirely Too Motivated

So this week I've been very motivated to work on my more nerdy hobbies. Here's an update on my latest progress

  • Made use of Hudson to set up a continuous build. WOW is this software great. I'm amazed at how good free software can get!

  • Attempted to get a Scala job up on hudson, so I'll know early on when scala fails on windows. :Sigh: I'm not sure what's going on here, but it looks like no one at EPFL really wants to maintain Scala for windows. While I understand their sentiments, I now have a mini-project at work that I cannot develop for because they've broken the eclipse plugin. I've managed to compile with an older scala, but this means I've lost my dev environment. I've heard someone say, if you stay on the bleeding-edge, expect to bleed. I guess I was hoping scala was further along in maturity then this. Oh well, back to contributing bug-fixes/enhancements in my spare time and secretly hoping we can use this for future development.
  • I managed to get Archiva up and running on ubuntu. For some reason (probably my stupidity), I had a difficult time figuring out where the common/lib file was. (Archiva requires derby, mail and activation in common/lib). For those of you like myself, common/lib is under /usr/share/tomcat5.5 NOT /var/lib/tomcat5.5. Anyway, now that I have archiva + hudson + trac up and running, it's time for a really sweet development environment. I'm working on trying to wire at least hudson into our project at work. I'll probably be posting what's useful and what to avoid later.



Also I'd like to take this time for a mini-rant.
<rant>
I'm really confused here. I got on the #eclipse-dev list serve to ask some questions about the eclipse PDE project. I received answers on where to look to solve my issues. I got on the #scala list serve to ask some general architecture questions, and was told to look at some haskell apps because Scala is impure (Note: this is my wording). Ok, the haskel App engine thing looked pretty cool, but I couldn't get past the syntax...(I'd rather read LOLCode all day). How many key contributors to Scala feel it's a "less-good" Haskell? I know a lot of libraries are inspired by Haskell, but not a lot of libraries appear to be inspired from other JVM languages. This confuses me, as I thought Scala was targeted towards Java developers. There aren't a lot of Helper routines to make using Scala with existing java code/libraries easier.

Example: I'm integrating with a bunch of Java-Threaded code. It uses Runnable and callable. I created a JavaThreadConversion object that looks like this:
object JavaThreadConversion
{
def functionToRunnable(f : => Unit) = new Runnable() {
override def run() = f
}
import java.util.conncurrent.Callable
def functionToCallable[A](f : => A) = new Callable() {
override def call() = f
}
}


That little bit of code save me a lot of typing when interfacing with my old java libraries.

Why don't you rewrite them in scala Not enough time/money/priority. Not gonna happen any time soon.

Oh well. Hopefully as more Java developers see the power of scala, they'll begin to contribute more utilities that can make transitioning off java easier. Until then, I'll see what of mine really is useful and can pass the "functional purity" filter.

Amusing note: At point I wondered why there weren't any Scala + OSGi helpers as I hear a lot of people asking how to do XYZ scala task using OSGi. I'd like to see how much that make a FP purist cringe... OSGi seems to be the bane of FP. Even worse would be the mixing of a Scala core with a Groovy plugin system...
</rant>

Anyway, don't mean to offend anyone. Just frustrated with certain things, including my slow-learning-process for various FP concepts (like monads)

5 comments:

meisjohn said...

So, I don't really get the whole continuous integration thing.

* Does it mean that cron will just do a daily update & commit? If so, why don't you just set up a cron job?

* What happens when the build fails? It seems that (with Hudson at least), you get an email with the build output including pass/fail but I don't really grasp the benefit of this.

At my work, we do an enormous amount of integration testing. Typically the build integration is done early and often by the developer him/herself and then a basic unit test is performed. Our process does include a weekly (manual) build amongst all of our constituent parts, for which it is given to the test engineers (of which I am one of many.) After some amount of time and maturity (usually based on schedule, not maturity) we build a controlled load (think official version).

Now that I've given context, are we effectively doing manually (in the build process) what Hudson will do for us? I started to read a write-up on CI but my interest started to wain half way through, so I'm asking you. :-)

I suppose it all depends on what your definition of "integration" is. In my mind, the exit criteria of integration includes above all else a confirmation that the system performs as desired. This truly requires hands-on testing and evaluation of what's being built.

<rant of my own> I hate the fact that I can't type normal html into the comments stuff like blockquotes & lists don't even work :-P </rant of my own>

J. Suereth said...

First of, sorry about the comments not taking HTML. However it can be useful
1) It prevents image spam
2) It's google blogger's default


Now, as to your continous integration question. It's nice to have the CI server build what's ACTUALLY in the svn repostiory. YOu can guarantee a "fresh" or "pristine" environment for the CI server. The other nice thing is you can run CI on more than one OS, so if someone commits non-portable code, it will email them with the fix.

You are right, CI is pretty much a shell script + Cron + web interface. However, it's a massive shell script + cron + web interface that I don't have to write AND looks pretty sleek. Oh, it will also host the build results, do diffs and let you look through/download build artifacts online. Once again, a bigger script than I'm willing to write.

Another benefit of CI (IMHO) is when combined with maven. I can have my CI create snapshot releases of projects into my maven repo. This way I'm always using up-to-date utility libraries without needed to download the project source. (I usually only do that if there's a larger bug that needs me to investigate further).

Another note: the name Continuous Integration is confusing. No it is not Integration testing (unless you have automated integration tests in your build, then it *is* integration testing). It's more integration in the sense that your build system completes based on what's in SVN. This is really helpful to catch developers who forgot to commit various files to the repository. The CI server could catch this BEFORE a co-worker, saving you both some hassle.

Ricky Clarkson said...

"it looks like no one at EPFL really wants to maintain Scala for windows"

As far as I can tell, you're complaining that a release candidate, including an Eclipse plugin that is in beta, are unstable. These things usually are unstable. That said, I believe EPFL is setting up a Windows CI server soon, so perhaps that will catch some of these.

J. Suereth said...

Perhaps you're correct. I've justice noticed a lack of windows support. I guess spaces-in-filepaths aren't a windows only thing, but it does mean I can't compile in my user directory on windows.

And yes, it was a rant, I needed to vent and it's over. I tend to exagerate when I'm ranting, so I do apologize if things are too out of hand. I still have a lot of respect for the Scala community/language and hope to see more in the future. Perhaps with more windows users, more windows bugs will get reported and therefore fixed.

Anyway, feel free to correct any other statements i may have made that are exagerated.

Joe said...

CI is the playground guard. Sure, the kids are all supposed to get along on their own, but CI makes sure that if there is a problem it isn't overlooked and the people that can resolve it are notified. It is the conscience of a development team (kinda).

I've used a CI process to build and deploy DEV and TEST environments, based on branches in the repository. Developers committed to DEV and it auto-deployed. When we wanted functional testers at it, we committed it to the TEST branch and CI did the rest. This way, the DEV/TEST websites were always reflected the repository without any "magic tweaks" that are hard to repeat. Production releases were still manual.

CI provides one central, consistent validator of the code, that let me sleep better at night as a project manager. You can configure integration testing, code compliance checks, deployment, webtests, FindBugs, or anything else you want to happen whenever code is changed. One key is that running off the repository enforces these team-wide checks without depending on the habits or religions of your developers.

Its continuous, because it can happen with each developer commit. But yes, in practicality it is a cron job that checks the repository for changes and then checkouts any changes and builds if necessary, or any other tasks you configured it to perform.