TODO: Sort out Jenkins

• Creating WebHooks for GitHub & Jenkins interaction.

Cf.

• Jenkins parameters
• Creating a Parameterised Build

• CI is important, people should realise this

• Get a list of Jenkins task we need & their frequency
• Backup / restore stratergy - use tinkBackup but beware it shutsdown Jenkins whilst its running
• List of accounts (& passwords)
• Common Jenkins design patters (Across iOS / Android)
• Installed tools job
  • node.js / newman / etc
  • Create a Jenkins job to install all the plugins needed for Jenkins
• Lockdown Jenkins accounts. R/O Accounts etc
• Move (Android) Jenkins instance to VM / Docker etc, running on Ubunto?

Remember, you may need to have an initial ssh -T git@github.com to use Jenkins and GitHub.

Parameterised Builds

See here

Setup

Notes on setting up an Android Mobile CI build server on a ‘fresh’ MacMini.

Password can be forced via a sudo passwd jenkins.

Local, here.

Plugins

• android-emulator-plugin & this
• Build / Flow
• Android and Jenkins

Technologies

• Continuous Integration (CI)
• Jenkins is a CI server.
• Android SDK
• Gradle is a the prefered tool for building Android projects.
• Appium
• Building, testing & Deploying Android Apps with Jenkins PDF

Prerequisits

MacMini running OS X El Capitan (10.11.x)

Basic Instalation

• homebrew
• brew install Caskroom/cask/java
• brew install jenkins
• brew install wget
• javac ->

Install Homebrew

ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"

More details over on GitHub and the brew homepage at brew.sh.

Install Java

By default Java isn’t installed, so open a terminal and enter java to request the install.

Follow the instructions.

When completed, you should see something like this:

    $ java -version
    java version "1.7.0_79"
    Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
    Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

Create a local jenkins user

Create an applications group

    dseditgroup -o create -n . -u username -p -r ‘Applications’ applications

Where username is the username of an Admin user (I used richard.watson)

get the id for that group

sudo dscl . -read /Groups/applications
# find a unique identifier to give the user
sudo dscl . -list /Users UniqueID
# create the jenkins user
sudo dscl . -create /Users/jenkins
sudo dscl . -create /Users/jenkins PrimaryGroupID 505
sudo dscl . -create /Users/jenkins UniqueID 1026
sudo dscl . -create /Users/jenkins UserShell /bin/bash
sudo dscl . -create /Users/jenkins RealName "Jenkins"
sudo dscl . -create /Users/jenkins NFSHomeDirectory /Users/jenkins
sudo dscl . -passwd /Users/jenkins
# create and set the owner of the home directory
sudo mkdir /Users/jenkins
sudo chown -R jenkins /Users/jenkins

Create a plist file

/Library/LaunchDaemons/homebrew.mxcl.jenkins.plist or here - ~/Library/LaunchAgents/homebrew.mxcl.jenkins.plist

Replace 127.0.0.1 with 0.0.0.0

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>homebrew.mxcl.jenkins</string>
    <key>ProgramArguments</key>
    <array>
    <string>/usr/bin/java</string>
    <string>-Dmail.smtp.starttls.enable=true</string>
    <string>-jar</string>
    <string>/usr/local/opt/jenkins/libexec/jenkins.war</string>
    <string>--httpListenAddress=127.0.0.1</string>
    <string>--httpPort=8080</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>UserName</key>
    <string>jenkins</string>
</dict>
</plist>

Should we restrict access to Jenkins Web server?

Starting & Stopping Jenkins

    launchctl load ~/Library/LaunchAgents/homebrew.mxcl.jenkins.plist
    launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.jenkins.plist

or (better)

    # first, install it:
    brew tap homebrew/services

    # then you can view the services:
    sudo brew services list

    # restarting:
    sudo brew services restart jenkins

Misc Jenkins

This could be needed for our homebrew install of Jenkins - Brewed Jenkins

Resources

• Advanced Jenkins for iOS & Mac
• Installing Jenkins on OS X Yosemite
• Ionic

install.sh

#!/bin/bash  

if [[ "$OSTYPE" == "darwin"* ]]; then
    # Install Homebrew
    ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

    brew install git
    # git config --global user.name "$GIT_USERNAME"
    # git config --global user.email "$GIT_EMAIL"

    # Install node + npm
    brew install node

    cd ~
    curl http://dl.google.com/android/android-sdk_r24.1.2-macosx.zip -o android-sdk.zip
    unzip android-sdk.zip
    rm android-sdk.zip

    echo 'export ANDROID_HOME=$HOME/android-sdk-macosx' >> ~/.bashrc
    echo 'export PATH=$PATH:$ANDROID_HOME/tools:$ANDROID_HOME/platform-tools' >> ~/.bashrc
    . ~/.bashrc
fi

Update the SDK

android update sdk --no-ui

Install Jenkins

brew install jenkins

Note: When using launchctl the port will be 8080.

To have launchd start jenkins at login: ln -sfv /usr/local/opt/jenkins/*.plist ~/Library/LaunchAgents Then to load jenkins now: launchctl load ~/Library/LaunchAgents/homebrew.mxcl.jenkins.plist Or, if you don’t want/need launchctl, you can just run: jenkins

We’ve had issues with Jenkins forgetting it’s DB. Look at this when you need reminding (Where whoami = ci):

lrwxr-xr-x  1 ci  staff  50 29 Mar 11:59 /Users/ci/Library/LaunchAgents/homebrew.mxcl.jenkins.plist -> /usr/local/opt/jenkins/homebrew.mxcl.jenkins.plist
-rw-r--r--@ 1 ci  staff  625  8 Apr 10:55 /Users/ci/Library/LaunchDaemons/homebrew.mxcl.jenkins.plist
-rw-r--r--  1 root  wheel  627  8 Apr 10:21 /usr/local/Cellar/jenkins/1.653/homebrew.mxcl.jenkins.plist

Note owners & timestamps in particular.

Parameters

A build can be started just by POSTing to it

http://server/job/myjob/buildWithParameters?PARAMETER=Value

Passing Parameter between stages in Git

Assuming that there is a jankins parameter called JIRA_COMMENT.

#!/bin/bash -l
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8

echo "JIRA_COMMENT is initially ${JIRA_COMMENT}"

export JIRA_COMMENT="Set JIRA_COMMENT from shell 1"

echo "JIRA_COMMENT is ${JIRA_COMMENT}"

java -jar ${JENKINS_HOME}/war/WEB-INF/jenkins-cli.jar  -s ${JENKINS_URL} set-build-parameter JIRA_COMMENT "${JIRA_COMMENT}"

echo "JIRA_COMMENT is now ${JIRA_COMMENT}"

Then…

#!/bin/bash -l
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8

echo "Second shell [${JIRA_COMMENT}]"

Notable plugins

Build Flow

The DSL defines the sequence of jobs to be built :

build( "job1" )
build( "job2" )
build( "job3" )

You can pass parameters to jobs, and get the resulting AbstractBuild when required :

b = build( "job1", param1: "foo", param2: "bar" )
build( "job2", param1: b.build.number )

Environment variables from a job can be obtained using the following, which is especially useful for getting things like the checkout revision used by the SCM plugin (‘P4_CHANGELIST’, ‘GIT_REVISION’, etc) :

def revision = b.environment.get( "GIT_REVISION" )

You can also access some pre-defined variables in the DSL :

• build the current flow execution
• out the flow build console
• env the flow environment, as a Map
• params triggered parameters
• upstream the upstream job, assuming the flow has been triggered as a downstream job for another job.

For example:

// output values
out.println 'Triggered Parameters Map:'
out.println params
out.println 'Build Object Properties:'
build.properties.each { out.println "$it.key -> $it.value" }

// use it in the flow
build("job1", parent_param1: params["param1"])
build("job2", parent_workspace:build.workspace)

Should we consider using build flows in Jenkins? An example would be something like this:

// See https://wiki.jenkins-ci.org/display/JENKINS/Build+Flow+Plugin
// For examples on how to use this
//

build ("android_parameterized_build", , param1: "develop")

retry ( 3 ) {
    build( "android_upload_to_hockey" )
}

To a large extend they are very similar to what we have (or could have) with fastlane.

Sample bash commands for Jenkins builds

#!/bin/bash -l

cd /Users/ci/.jenkins/workspace/TestNightlyFastlane/src
xcodebuild test -workspace MY-WORKSPACE.xcworkspace -scheme Nightly -destination 'platform=iOS,name=iPhone 6s'
#FASTLANE_EXPLICIT_OPEN_SIMULATOR=1 scan --scheme "Nightly" --device "iPhone 6s"

Code Signing hell, with help from FixCode

http://www.mokacoding.com/blog/packaging-swift-ipa/ https://gist.githubusercontent.com/phatblat/6eb8895e2202f796960e/raw/264765c05ecc8918425cb513b12409de91e3a675/package-ida.sh

Use Jenkins to build Jenkins

e.g. have a install-prerequisits job which does stuff like this:

npm install -g newman

newman

FastLane & Jenkins & here

Jenkins

Currently installed on Mac - should move to a Docker image.

Start Jenkins:

sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist

Stop Jenkins:

sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist or
ps -e | grep jenkins
53 ?? 0:02.51 /usr/bin/java -jar /Applications/Jenkins/jenkins.war 392 ttys000 0:00.00 grep jenkins
$ sudo kill 53

Jenkins URLs

• http://www.vogella.com/tutorials/Jenkins/article.html#androiddescription_prereqs
• iOS CI with Jenkins

Disallow spaces in job names \S*

Jenkins Best Practices

• https://en.wikipedia.org/wiki/Continuous_integration<- Read this!
• https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+Best+Practices
• http://www.slideshare.net/andrewbayer/7-habits-of-highly-effective-jenkins-users

Set up version control of job configurations

Keep jobs simple! Don’t put a ton of bash in each job. If a job needs to do something complex, put it in a script in GitHub and check it out as needed.

Use templated builders to simplify common tasks

Keep all scripts in version control - avoid running scripts that live on the Jenkins server filesystem Don’t install unnecessary plugins - plugins are often written by third parties and can interact with each other in strange ways Use LDAP authentication if possible for traceability - avoid anonymous access If possible, set up user/group roles so that only the people who need access to the internals of Jenkins have access

Backup artifacts

Keep Jenkins up to date! Out of date installations tend to rot. Tie jobs into Sonar to track code quality Commit often! It’s easier to collaborate on code when everyone is constantly integrating their changes. Job names in Jenkins should not use spaces - directories are created for each job, and some tools don’t support directories with spaces You can enforce this with the Restrict Project Naming global configuration using the pattern \S* Set up a Jenkins display where everyone can see when your jobs are failing

Guy’s Favorite Jenkins Plugins

Doony: Not quite a plugin, but it’s a set of easy-to-install UI improvements for Jenkins. http://doony.org/ AnsiColor: Colorized console logs. Build Monitor Plugin: Pretty live job status monitor. Build Trigger Badge Plugin: At a glance, see why each build was triggered - an upstream job, a manual trigger, an SCM trigger, etc. Console Tail Plugin: See the last n lines of the latest build on a job’s main page. Find out why your build failed without digging through logs. embeddable-build-status: Show your build status with a live icon you can embed in your GitHub readme. Jenkins disk-usage plugin: Easily keep track of your disk space usage on the Jenkins master. Jenkins Jabber notifier plugin: Notify chatrooms and users of build failures. Jenkins Workspace Cleanup Plugin: Delete your workspace before or after builds. Ensure a fresh build every time. Monitoring: Track Jenkins master and slave performance. This is crucial when your Jenkins installation starts acting funny and you want to know why. Naginator: Automatically re-run failed builds, in case they failed due to an environmental issue. ShiningPanda Plugin: Creates Python virtualenvs for you to work in. Template Project plugin: Use jobs as templates for other jobs. Lets you easily manage multiple jobs that share a similar configuration. Timestamper: Timestamp your console logs. SCM Sync Configuration Plugin: Sync your Jenkins configuration files (which are mostly XML) to GitHub. Version control everything.

## Jenkins Administration

If you need to restart Jenkins, go to /safeRestart or /restart If this doesn’t work, you’ll need to SSH into Jenkins and run sudo service jenkins restart To update Jenkins, run sudo yum update jenkins If you install plugins, do it one at a time and restart Jenkins after each plugin, then go into a few configuration menus and run a few jobs to make sure nothing is broken. Installing Jenkins plugins is a bit risky and it’s best avoided unless you have a really good reason to do it. We’re using the latest LTS Jenkins version. https://wiki.jenkins-ci.org/display/JENKINS/LTS+Release+Line The Jenkins home directory is /var/lib/jenkins. Don’t disable the Maven plugin. Jenkins won’t start up. If you do disable it by accident, re-enable it with sudo rm /var/lib/jenkins/plugins/maven-plugin.jpi.disabled If you accidentally lock everyone out of Jenkins due to incorrect security settings, disable global security in /var/lib/jenkins/config.xml

Jenkins & Docker

Run ‘Docket Quickstart Terminal’, then eval $(docker-machine env)

Then docker pull jenkins to get the TLS image.

docker run -p 8080:8080 jenkins does the magic, more details here.

docker run -d -p 49001:8080 -v $PWD/jenkins:/var/jenkins_home -t jenkins