Tim On Computer Science

This article was inspired by a strangely acting Gradle build script. (Unimportant details: it worked fine locally, and it worked in Jenkins the first time, but when I renamed the job, it mysteriously started failing with a compile error every time after that.)

As I was troubleshooting that Jenkins job, I had a thought – I’ve known engineers and one manager who would have “just fixed it” – they would have changed the job name back to what worked and moved on. In the case of the (ex) manager, he would have made a decree to just change the name back, and would have been angry if anybody dared question his command.

I used to be annoyed by that manager – until I realized two things: (1) He does not have any real choice at this stage in his career. He has chosen the easy path for so long, he literally never developed the skills needed to understand and solve difficult problems. And since he doesn’t have the skills, he assumes nobody else does either, or gets jealous when they demonstrate those skills, and (2) I have chosen a different path. I think of it as “no magic in my world”. As in, until I figured it out, it was “magic” why one Jenkins job worked yet the other failed, and “magic” is not allowed as an explanation in my world. (I also sometimes think of this as “never be afraid of the system you are developing”.) And yes, I figured it out, and learned another piece of the Gradle system. There is neither “magic” nor fear left in that particular area for me.

A thought-provoking question: which Path is more efficient? I can’t say for sure – because when I can rattle off an answer to the current problem in 2 seconds, that feels good, and feels productive, but it took years to learn where to put the X. Either pay gradually over time or gradually over time become afraid.

I do know one thing though: when push comes to shove, I’m capable of acting like (and I have numerous times acted like) I’ve followed the “head in the sand” path. The opposite is not true, however.

For those of you on the “no magic” path – the “X” is: Do not use project.name as part of your generated sources. Any time the directory name changes (as in Jenkins, which creates the workspace directory name from the Jenkins job name), the value of Gradle’s project.name will change too.

For those of you on the other path – if you are reading this, then there is still hope for you yet.

This shows the way to get gradle to use a local Archiva server to publish your project artifacts.

IMPORTANT: You must configure your “guest” user in Archiva to have the correct permissions (roles) to publish to the Archiva server. To do this, in the Archiva WebUI (the default is http://localhost:8080), on the left side under “USERS”, click “Manage”, then click the blue pencil next to “guest”, then click the “Edit Roles” (next to the blue “Edit” button), then checkbox select “Global Repository Manager” and “Global Repository Observer” and press Update.

Interesting fact: Archiva will accept “SNAPSHOT” artifacts into its “internal” repository (i.e. the upload will succeed, and Archiva will store the artifacts in its “/repositories/internal/” directory). BUT it will not serve these artifacts, saying instead “managed repo is configured for release only”. If you accidentally publish SNAPSHOT artifacts to the “internal” repository, then you’ll have to clean your “/repositories/internal/” directory by hand – the WebUI won’t let you.

So, given that interesting fact, the “if” logic (below) looks for “SNAPSHOT” and sets the url to the correct location. Note that the single “maven { }” entry is different from how it was configured for resolving, where there were two entries. (See Gradle with Local Archiva).

This shows the correct ‘publishing’ section of your build.gradle file for publishing to your local Archiva server. This also shows how to publish your “-sources” artifact. Feel free to substitute “localhost” for the actual machine name or IP address of your Archiva server.

(2014/6/10 update: added ext.isReleaseVersion and added checks before using the .publishBaseUrl property before using it.)

Versions:
archiva: 2.0.1
gradle: 1.11

// Used in publishing - the new plugin:
apply plugin: 'maven-publish'

// Used in publishing - pom information: 
group =  'com.tiemens'
version =  '0.1-SNAPSHOT'
project.ext.isReleaseVersion = !version.endsWith("SNAPSHOT")

// Used in publishing - source artifact: 
task sourceJar(type: Jar) {
    from sourceSets.main.allJava
}

publishing {
    publications {
        mavenJava(MavenPublication) {

            from components.java

            artifact sourceJar {
                classifier "sources"
            }
        }
    }

    repositories {
        maven {
            if (project.hasProperty('publishBaseUrl')) {
                if (! project.ext.isReleaseVersion) {
                    url project.publishBaseUrl + "/snapshots"
                } else {
                    url project.publishBaseUrl + "/internal"
                }
            } else {
                    // this is a notice that 'publish' requires .publishBaseUrl
                    url "http://you.must.configure.project.publishBaseUrl"
            }
        }
    }

}

Just for documentation, this shows the error message you get if you don’t set up your guest user with the proper roles.

$ gradle --debug publish
... snip ...
17:09:38.693 [DEBUG] [sun.net.www.protocol.http.HttpURLConnection] sun.net.www.MessageHeader@19507226 pairs: {PUT /repository/snapshots/com/tiemens/CardWar/0.5-SNAPSHOT/CardWar-0.5-20140421.220938-1.jar HTTP/1.1: null}{User-Agent: maven-artifact/3.0.4 (Java 1.7.0_45; Linux 3.2.0-31-generic-pae)}{Host: sitearchiva:8080}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}{Content-Length: 21915}
17:09:38.703 [DEBUG] [sun.net.www.protocol.http.HttpURLConnection] sun.net.www.MessageHeader@10775136 pairs: {null: HTTP/1.1 401 Unauthorized}{Date: Mon, 21 Apr 2014 22:09:38 GMT}{Set-Cookie: JSESSIONID=11ohtcvqtpqu3s9r3iis2a7c1;Path=/}{WWW-Authenticate: Basic realm="Repository Archiva Managed snapshots Repository"}{Content-Length: 0}{Server: Jetty(8.1.14.v20131031)}
17:09:38.737 [INFO] [org.gradle.api.internal.project.ant.AntLoggingAdapter] [ant:null] An error has occurred while processing the Maven artifact tasks.
Diagnosis:

Error deploying artifact 'com.tiemens:CardWar:jar': Error deploying artifact: Failed to transfer file: http://sitearchiva:8080/repository/snapshots/com/tiemens/CardWar/0.5-SNAPSHOT/CardWar-0.5-20140421.220938-1.jar. Return code is: 401
...snip...


More documentation links:

Shows example using authentication, instead of using “guest” with publish role:
http://forums.gradle.org/gradle/topics/maven_publish_and_setting_snapshotrepository_and_releaserepository

This shows the way to get gradle to use a local Archiva server to resolve dependencies (where the Archiva server in turn resolves to maven central).

(Side comment: documentation for gradle is sparse, incomplete and out-of-date – e.g. ‘mavenRepo’ shows up in many searches.)

So, if you’re interested in how to use a local Apache Archiva server in your environment, then this shows the correct ‘repositories’ section of your build.gradle file. In an upcoming post, I’ll document how to publish to your local Archiva server. Feel free to substitute “localhost” for the actual machine name or IP address of your Archiva server.

Versions:
archiva: 2.0.1
gradle: 1.10

repositories {
   maven {
     url 'http://localhost:8080/repository/internal'
   }
   maven {
     url 'http://localhost:8080/repository/snapshots'
   }

  // if your Archiva is set up correctly, 
  // then you don't need mavenCentral() here:
  //  mavenCentral()
}

Just for documentation, this shows the “dependencies” section of the build.gradle file. If your Archiva is set up correctly, these 3 dependencies will actually download/store 8 .jar files (because of transitive dependencies).

dependencies {
  testCompile group: 'info.cukes',   name: 'cucumber-java',    version: '1.1.5'
  testCompile group: 'info.cukes',   name: 'cucumber-junit',   version: '1.1.5'
  testCompile group: 'junit',        name: 'junit',            version: '4+'
}

Files that end up in your apache-archiva/repositories directory (showing just the .jar files):

apache-archiva-2.0.1/repositories/internal/junit/junit/4.11/junit-4.11.jar
apache-archiva-2.0.1/repositories/internal/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar
apache-archiva-2.0.1/repositories/internal/info/cukes/cucumber-html/0.2.3/cucumber-html-0.2.3.jar
apache-archiva-2.0.1/repositories/internal/info/cukes/cucumber-java/1.1.5/cucumber-java-1.1.5.jar
apache-archiva-2.0.1/repositories/internal/info/cukes/gherkin/2.12.1/gherkin-2.12.1.jar
apache-archiva-2.0.1/repositories/internal/info/cukes/cucumber-junit/1.1.5/cucumber-junit-1.1.5.jar
apache-archiva-2.0.1/repositories/internal/info/cukes/cucumber-jvm-deps/1.0.3/cucumber-jvm-deps-1.0.3.jar
apache-archiva-2.0.1/repositories/internal/info/cukes/cucumber-core/1.1.5/cucumber-core-1.1.5.jar

Documentation – The good stuff:
http://www.gradle.org/docs/1.8-rc-1/release-notes.html — shows the change from mavenRepo to maven.
http://gradleproject.wordpress.com/2013/02/14/multiple-maven-repositories-in-gradle/ – now NOT to set up your repositories entry.

More Documentation – The list of things that don’t work:
http://gradle.1045684.n5.nabble.com/using-Archiva-maven-repo-w-Gradle-td4579298.html
http://joshdiehl.com/2011/07/11/using-apache-archiva-with-gradle/

I’ve always enjoyed finding bugs in languages. I still fondly recall finding a bug in Perl concerning the “while () { .. }” construct and file names that evaluate to ‘false’, like a file named “0”, for example.

More recently, I discovered a Groovy bug in groovy.lang.SpreadMap class (and therefore in the List.toSpreadMap() implementation). I’ve reported it (GROOVY-6403) and submitted a patch with a unit test.

Some observations:

• This implementation was horribly broken. Two seconds of review is enough for competent Java developers to see why.
• This code has been around a long time. This Groovy Goodness post is dated January, 2010. It shows the two methods (.size() and .get()) that were implemented correctly. I’ll wonder forever if all the other broken methods (.containsKey(), .keySet(), .isEmpty(), etc.) were missed or purposefully omitted.
• This code could have been easily detected as flawed with a 5-line unit test.

The original example, with some additional (failing) asserts:

#!/usr/bin/env groovy
def list = ['key', 'value', 'name', 'tim'] as Object[]
def map = list.toSpreadMap()

assert 2 == map.size()                // ok
assert 'value' == map.key             // ok
assert true == map.containsKey('key') // FAIL
assert false == map.isEmpty()         // FAIL
assert 2 == map.keySet().size()       // FAIL

Back to the Perl bug

For these code snippets, place files named “0”, “1” and “2” in the current directory. Perl 5 was used for this testing.

The one will print “File is 0”, “File is 1” and “File is 2”, but also prints “How did we get here if 0 is not true?” [In earlier versions of Perl, this would not print anything if the first file found was “0”. They’ve changed that. But, see the next example.]

#!/usr/bin/env perl
while ($file = ) {
   print "File is $file\n";
   if (! $file) {
      print "Huh?  How did we get here if $file is not true?\n";
   }
}

This one just won’t print anything, since “0” is the first file it sees, and that evaluates to ‘false’. The extra variable assignment causes the hack fix to fail:

#!/usr/bin/env perl
my $inside;
my $file;
while ($file = ($inside = )) {
   print "File is $file\n";
}

P.S. In case it is not obvious, do not use either of these constructs. In summary:

• For the Perl bug, use opendir() and readdir().
• For the Groovy bug, look at what the constructor of SpreadMap.java does, and write your code that way. Or, use Groovy 2.2.0-rc-3+ or 2.1.10+