fd Blog

Daniel Hilgarth on software development

Simple Firewall Setup on Ubuntu

Install iptables-persistent:

sudo apt-get install iptables-persistent

Create the firewall rules:

sudo nano /etc/iptables/rules.v4

Use these rules as a starting point:

*filter
:INPUT DROP
:FORWARD DROP
:OUTPUT ACCEPT

# Allows all loopback (lo0) traffic and drop all traffic to 127/8 that doesn't use lo0
-A INPUT -i lo -j ACCEPT
-A INPUT ! -i lo -d 127.0.0.0/8 -j REJECT

# Allow established connections
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allows all outbound traffic
-A OUTPUT -j ACCEPT

# Allows HTTP and HTTPS connections from anywhere (the normal ports for websites)
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT

# Allow FTP connections from anywhere
-A INPUT -p tcp --dport 21 -j ACCEPT
-A INPUT -p tcp --destination-port 49152:65534 -j ACCEPT

# Allows SSH connections
## SSH
-A INPUT -p tcp --dport 22 -j ACCEPT

# Allow ping
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT

# log iptables denied calls (access via 'dmesg' command)
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7

COMMIT

Move TeamCity Installation to New Server

On the old server

Create a backup

  1. Log into TeamCity with an account with administrative rights.
  2. Navigate to Administration > Server Administration\Backup
  3. On the “Run Backup” tab, change the “Backup scope” to “Custom” and select everything.
  4. Stop TeamCity

     sudo service teamcity stop
    
  5. Transfer the data directory to the new server:

     scp -P 22 -r /opt/jetbrains/TeamCity/.BuildServer <user>@<new-server>:/location/on/new/server
    

    This will also transfer the backup file.

On the new server

Setup TeamCity

As per the first part of this post. Additionally, follow the steps to create the MySQL database.

Prepare for backup restore

sudo mkdir /opt/jetbrains/TeamCity/.BuildServer/
sudo mkdir /opt/jetbrains/TeamCity/.BuildServer/lib/
sudo mkdir /opt/jetbrains/TeamCity/.BuildServer/lib/jdbc/
sudo mv /location/on/new/server/.BuildServer/lib/jdbc/mysql-connector-java-5.1.31-bin.jar /opt/jetbrains/TeamCity/.BuildServer/lib/jdbc/
sudo chown teamcity:teamcity /opt/jetbrains/TeamCity/.BuildServer/lib/jdbc/mysql-connector-java-5.1.31-bin.jar

Restore the backup

sudo -i
cd /opt/jetbrains/TeamCity/bin
./maintainDB restore -F /location/on/new/server/.BuildServer/backup/<backup-file> -A /opt/jetbrains/TeamCity/.BuildServer -T /location/on/new/server/.BuildServer/config/database.properties
exit
sudo rm -rf /opt/jetbrains/TeamCity/.BuildServer/system/artifacts
sudo cp -r /location/on/new/server/.BuildServer/system/artifacts /opt/jetbrains/TeamCity/.BuildServer/system/
sudo cp -r /location/on/new/server/.BuildServer/plugins /opt/jetbrains/TeamCity/.BuildServer
sudo cp -r /location/on/new/server/.BuildServer/system/pluginData /opt/jetbrains/TeamCity/.BuildServer/system
sudo chown -R teamcity:teamcity /opt/jetbrains/TeamCity

Start TeamCity

sudo service teamcity start

References

http://blogs.endjin.com/2013/07/a-step-by-step-guide-to-migrating-a-teamcity-instance-from-one-server-to-another/

TeamCity 8: Setup on Linux

Introduction

This tutorial shows how to install TeamCity on a Linux server. Please note that you will need a Build Agent that runs on Windows if you want to build .NET projects.

Use Apache as Proxy for Confluence, JIRA & Stash

This tutorial shows how to use Apache as a proxy for Confluence, JIRA & Stash.
We will configure Apache so that it only accepts SSL connections, but without the need to enable SSL in the TomCat instances of Confluence, JIRA & Stash.
Each application will be available at its own sub domain, e.g. confluence.<your-domain>.com etc.

Apache configuration

Site configuration

For each application, create a config file in /etc/apache2/sites-available with the following content:

<Virtualhost *:80>
  ServerName <sub>.<domain>
  Redirect permanent / https://<sub>.<domain>/
</Virtualhost>

<Virtualhost *:443>
  ServerName <sub>.<domain>
  SSLEngine On
  SSLProxyEngine On
  ProxyPass        /  http://localhost:<port>/ connectiontimeout=5 timeout=300
  ProxyPassReverse /  http://localhost:<port>/
</Virtualhost>

Where <sub> is the sub-domain, e.g. confluence, <domain> is your domain, e.g. fire-development.com and <port> is an unused port. Please note, this is not the default port of the application, i.e. it is not 8080 for confluence. Port and sub-domain need to be different in each file you create. I used 8443, 8444 and 8445 for the ports and confluence, jira and git for the sub-domains.

Enable the sites via sudo a2ensite <config-file>

Required modules

Enable the required modules:

sudo a2enmod ssl
sudo a2enmod proxy
sudo a2enmod proxy_http

SSL configuration

Create a new file /etc/apache2/conf-available/ssl.conf with the following content:

SSLCertificateFile /etc/apache2/ssl/<certificate>
SSLCertificateKeyFile /etc/apache2/ssl/<certificate-key>
SSLCACertificateFile /etc/apache2/ssl/<intermediate-certificate>

Example:

SSLCertificateFile /etc/apache2/ssl/rapidssl-fire-development.com.crt
SSLCertificateKeyFile /etc/apache2/ssl/rapidssl-fire-development.com.key
SSLCACertificateFile /etc/apache2/ssl/rapidssl-intermediate.crt

Make sure the referenced files actually exist :-)

Finally, you need to enable the new configuration file:

sudo a2enconf ssl

TomCat configuration

For each application, you need to edit the file path/to/application/conf/server.xml, e.g. sudo nano /opt/atlassian/confluence/conf/server.xml.

In each file, you need to find the existing Connector tag. It should be inside the Server/Service tag. Right below it, still inside the Service tag, insert another Connector tag. <port>, <sub> and <domain> are the values from the corresponding config file created in the first step.

Confluence

<Connector className="org.apache.coyote.tomcat4.CoyoteConnector" port="<port>" minProcessors="5"
           maxProcessors="75"
           enableLookups="false" acceptCount="10" debug="0" connectionTimeout="20000"
           useURIValidationHack="false" URIEncoding="UTF-8"
           scheme="https"
           proxyName="<sub>.<domain>"
           proxyPort="443"/>

JIRA

<Connector port="<port>"

           maxThreads="150"
           minSpareThreads="25"
           connectionTimeout="20000"

           enableLookups="false"
           maxHttpHeaderSize="8192"
           protocol="HTTP/1.1"
           useBodyEncodingForURI="true"
           acceptCount="100"
           disableUploadTimeout="true"
           scheme="https"
           proxyName="<sub>.<domain>"
           proxyPort="443"/>

Stash

<Connector port="<port>" protocol="HTTP/1.1"
           connectionTimeout="20000"
           useBodyEncodingForURI="true"
           compression="on"
           compressableMimeType="text/html,text/xml,text/plain,text/css,application/json,application/javascript,application/x-javascript"
           scheme="https"
           proxyName="<sub>.<domain>"
           proxyPort="443"/>

Apply changes

To apply the changes, all involved applications need to be restarted:

sudo service confluence stop
sudo service confluence start
sudo service jira stop
sudo service jira start
sudo service atlstash stop
sudo service atlstash start
sudo service apache2 restart

Fix Base URL

Each application has a configuration value that specifies the Base URL that users use to access it. This value needs to be adjusted to https://<sub>.<domain>, e.g. https://confluence.fire-development.com.

Please refer to the Atlassian documentation on how to do this:

Move Simple Wordpress Installation to New Server

On the old server

Create a database dump

sudo mysqldump -p wordpress > wordpress.dump

Copy data to new server

Copy the following to the new server:

  • database dump
  • Wordpress installation directory
  • Apache configuration

You can do this using these commands:

scp  wordpress.dump <user>@<new-server>:/location/on/new/server
scp -r /var/www/<wordpress-folder> <user>@<new-server>:/location/on/new/server 
scp /etc/apache2/sites-enabled/<config-file> <user>@<new-server>:/location/on/new/server  

On the new server

Create empty database

sudo mysql -p
create database wordpress;
grant all privileges on wordpress.* TO "wordpressuser"@"localhost" identified by "<password>";
flush privileges;
exit;

Restore the database

sudo mysql -p wordpress < wordpress.dump

Place the files in the correct folder

sudo mv /location/on/new/server/<wordpress-folder> /var/www/
sudo mv /location/on/new/server/<config-file> /etc/apache2/sites-available

Fix file permissions

sudo chown -R www-data:www-data /var/www/<wordpress-folder>
sudo chown root:root /etc/apache2/sites-available/<config-file>

Enable the site in apache

sudo a2ensite <config-file>
sudo service apache2 restart

Prepare Ubuntu for Wordpress Installation

These instructions detail how to prepare a new Ubuntu 14.04 LTS server for a Wordpress installation.

Install Apache

sudo apt-get install apach2

Install PHP

sudo apt-get install libapache2-mod-php5

Install MySQL

sudo apt-get install mysql-server libapache2-mod-auth-mysql php5-mysql

Configure Apache

## Enable PHP

sudo a2enmod php5

Improve security

Disable sending Apache and Ubuntu version

sudo nano /etc/apache2/conf-enabled/security.conf

Change ServerSignature and ServerTokens to these values:

ServerSignature Off
ServerTokens Prod

Disable directory listing

sudo nano /etc/apache2/apache2.conf

Find the entry Options Indexes FollowSymLinks directly beneath <Directory /var/www/>. Change it to:

Options -Indexes +FollowSymLinks

Restrict access to directories

sudo nano /etc/apache2/conf-enabled/security.conf

Activate these lines by uncommenting them:

<Directory />
   AllowOverride None
   Order Deny,Allow
   Deny from all
</Directory>

Disable unnecessary modules

sudo a2dismod autoindex
sudo a2dismod status
sudo a2dismod include

Disable default sites

sudo a2dissite 000-default
sudo a2dissite default-ssl

Restart Apache

sudo service apache2 restart

References

  • http://www.tecmint.com/apache-security-tips/

Install Java on Linux

The following instructions detail how to install Java (JRE or JDK) on a Ubuntu server.

  1. Get the download URL of the .tar.gz archive of the version you want. If you want to install the JRE, you can get the URL from here. For the JDK, you can get it from here. For the JDK, make sure you first accept the license agreement, before getting the URL from the page.

  2. Download the file
    • For the JRE:

      wget <URL> -O jre.tar.gz
      
    • For the JDK:

      wget --no-cookies --no-check-certificate --header "Cookie: oraclelicense=accept-securebackup-cookie" "<URL>" -O jdk.tar.gz
      
  3. Extract the downloaded file:
    • JRE:

      tar zxvf jre.tar.gz
      
    • JDK:

      tar zxvf jdk.tar.gz
      
  4. Move the extracted folder to /usr/lib/jvm. Replace jre1.7.0_65 with the actual folder that has been created by the extraction.

    sudo mkdir -p /usr/lib/jvm ; sudo mv jre1.7.0_65/ /usr/lib/jvm
    
  5. Actually install Java:
    1. Get the existing installations: sudo update-alternatives --config java. This will give a list of choices:

      $ sudo update-alternatives –-config java
      There are 2 choices for the alternative java (providing /usr/bin/java).
      
      Selection Path Priority Status
      
      0 /usr/lib/jvm/java-6-openjdk/jre/bin/java 1061 auto mode
      1 /usr/lib/jvm/java-6-openjdk/jre/bin/java 1061 manual mode
      2 /usr/lib/jvm/java-6-sun/jre/bin/java 63 manual mode
      

      Remember the greatest number and add 1, so in this example, you would remember 3.

      If you don’t get a list of choices you probably get an error message:

      update-alternatives: error: no alternatives for java

      That’s fine, you don’t have java installed so far. Just remember 1 in this case.

      Or you might get:

      There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/jdk1.7.0_67/bin/java Nothing to configure.

      In that case, remember 2.

    2. Now, execute:

      sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/jre1.7.0_65/bin/java <number you remembered>
      

      Again, replace jre1.7.0_65 with the actual folder name.

    3. If you are installing the JDK, execute the following, too:

      sudo update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/jre1.7.0_65/bin/javac <number you remembered>
      
    4. Fix the ownership and permissions:

      sudo chmod a+x /usr/bin/java
      # If you are installing the JDK:
      # sudo chmod a+x /usr/bin/javac
      sudo chown -R root:root /usr/lib/jvm/jre1.7.0_65
      
    5. Set JAVA_HOME environment variable:

      sudo nano /etc/profile.d/java.sh
      

      Add the following lines at the end of the file:

      export JAVA_HOME=/usr/lib/jvm/jre1.7.0_65
      export PATH=$JAVA_HOME/bin:$PATH
      

      If the file didn’t exist yet, then that’s perfectly fine. Just create it and set the correct ownership and permissions:

      chown root:root /etc/profile.d/java.sh
      chmod 755 /etc/profile.d/java.sh
      
    6. Verify Java has been installed correctly by executing java -version:

      $ java -version
      java version "1.7.0_25"
      Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
      Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
      
    7. You might get the following error:

      -bash: /usr/bin/java: No such file or directory

      In this case you probably installed the 32 bit version of Java on a 64 bit OS.
      You should be able to solve this problem by executing:

      sudo apt-get install libc6-i386
      

      After the installation finished java -version should work now.

References:

Setting Up SSH Port Forwarding for Atlassian Stash

To setup SSH port forwarding for Atlassian Stash on Ubuntu, perform the following steps:

  1. Change default port for SSH access in the file /etc/ssh/sshd_config to something other than 22.
  2. Restart SSH:

     # sudo restart ssh
    
  3. Make sure to allow the new SSH port in the firewall
  4. Install haproxy:

     # sudo apt-get install haproxy
    

Setting Up PostgreSQL for Confluence

These are the steps to setup a new Confluence database or to move a Confluence database to a new server. The steps are the same for other Atlassian products. The only difference is the role name and the database name.

How to Register a MSSCCI Source Control Plugin on a 64 Bit Machine

More a reminder for myself than anything else:

To register a MSSCCI source control plugin (nowadays called Source Control Plug-in) on a 64 bit machine, the instructions in the MSDN are missing some crucial information:
All registrations in HKLM\Software need to be performed inside HKLM\Software\Wow6432Node

The full file to register a sample plugin looks like this:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\SourceCodeControlProvider\InstalledSCCProviders]
"MSSCCI Sample"="Software\\Your Company\\Your Product"

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Your Company]

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Your Company\Your Product]
"SCCServerName"="MSSCCI Sample"
"SCCServerPath"="D:\\Temp\\SampleMSSCCI\\Debug\\SampleMSSCCI.dll"

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\SourceCodeControlProvider]
"ProviderRegKey"="Software\\Your Company\\Your Product"

[HKEY_LOCAL_MACHINE\SOFTWARE\SourceCodeControlProvider\InstalledSCCProviders]
"MSSCCI Sample"="Software\\Your Company\\Your Product"

[HKEY_LOCAL_MACHINE\SOFTWARE\Your Company]

[HKEY_LOCAL_MACHINE\SOFTWARE\Your Company\Your Product]
"SCCServerName"="MSSCCI Sample"
"SCCServerPath"="D:\\Temp\\SampleMSSCCI\\Debug\\SampleMSSCCI.dll"

[HKEY_LOCAL_MACHINE\SOFTWARE\SourceCodeControlProvider]
"ProviderRegKey"="Software\\Your Company\\Your Product"

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\8.0\CurrentSourceControlProvider]
@="{53544C4D-FFD5-48A9-B04A-0700CCD31070}"
"ProviderRegKey"="Software\\Your Company\\Your Product"

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\8.0\SourceControl]
"WarnedOldMSSCCIProvider"=dword:00000500
"UseOldSCC"=dword:00000001

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\8.0Exp\SourceControl]
"WarnedOldMSSCCIProvider"=dword:00000500
"UseOldSCC"=dword:00000001

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\8.0Exp\CurrentSourceControlProvider]
@="{53544C4D-FFD5-48A9-B04A-0700CCD31070}"
"ProviderRegKey"="Software\\Your Company\\Your Product"

Setup a Virtual Machine With Ubuntu Server

Setting up a virtual machine with Ubuntu Server is very simple.

  1. Download VMware Player
  2. Download Ubuntu Server ISO. I downloaded Ubuntu Server 13.04.
  3. Create a new virtual machine using VMware Player:
    1. Select “Create A New Virtual Machine”
    2. Choose “Installer disc image file” and select the downloaded ISO. It should detect the Ubuntu version and offer “Easy Install”.
    3. Enter your name and the details of the initial user account
    4. Provide a name and location for the virtual machine
    5. Start installation

Install Xamarin in vs 2013 Preview 1

As far as I know, Xamarin hasn’t yet released an installer for Visual Studio 2013 Preview 1.
Until they do, you can follow these simple instructions:

  1. Install Xamarin in Visual Studio 2012
  2. Copy the files from c:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Xamarin\ to c:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\Xamarin\
  3. Copy the folders Android and ios from inside c:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\ProjectTemplates\CSharp\ to c:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\ProjectTemplates\CSharp\
  4. Copy the folders Xamarin.Android and ios from c:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\ItemTemplates\CSharp\ to c:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\ItemTemplates\CSharp\
  5. Execute the following command from a command prompt with elevation: "c:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\devenv.exe" /setup /nosetupvstemplates

That’s it. You should now have a fully functional Xamarin setup in VS 2013!

Unit-testing With AutoFixture, NCrunch and Global State

Trying to unit-test code that relies on global state can be a frustrating experience. Global state is shared among tests, making the tests no longer independent. This can lead to subtle bugs in the test suite, e.g. a scenario where tests seem to fail randomly when executed in bigger batches, but pass when executed alone. Tracking down - and eliminating - the state sharing can be both time consuming and frustrating.

A Fluent Repository: Immutability

It has been pointed out that the implementation of the Fluent Repository base class I showed in Part 2 has a big flaw:
You need a new instance of the repository for each query, because the base class is mutable and thus it appends the query conditions to the internally stored IQueryable<T>. That’s not obvious and in fact counter-intuitive and because of that, it is bad.

A Fluent Repository Implementation - Part 1

Problem with classical implementations of the Repository pattern

After establishing that the Repository pattern is still the pattern of choice for separating the domain layer from the data access layer, I want to talk about a common problem when implementing it: Method explosion.

Why the Repository Pattern Is Still Valid

Introduction - what is the Repository pattern?

The Repository pattern has been around for quite a while. It was described as follows:

Mediates between the domain and data mapping layers using a collection-like interface for accessing domain objects.

The Repository pattern is a powerful pattern to separate the domain layer from the concrete implementation of the data access layer (DAL).

Automatic Syntax Highlighting

In my last post about the syntax highlighting of this blog I explained how to get Google’s code prettify to work.

Both options presented require me to add some kind of marker to the code blocks. That quickly started to annoy me. What I really wanted is the highlighting to happen without any markers, just like on Stack Overflow.

Blog Publishing

The last article in the series about the blog itself shows how I actually publish the generated content.

Blogging Workflow

One goal for when actually writing my posts was low friction. I didn’t want to execute ten different commands to just commit the fix of a typo.

Blog Organization

One thing I liked very much when I started exploring the world of static blog generators was the ability to have the raw data (the markdown) of all of my posts in a git repository. Nicely versioned and all.
But I didn’t want this repository to be public - no one really needs to see the intermediate states of my posts. Additionally, I didn’t want my posts to be in that huge Octopress repository.

Blog Setup

As many others starting a blog with a static blog generator, I will dedicate the first posts to talking about my setup.

When I decided to finally start a blog, I had one main requirement the blogging engine had to fulfill:
I wanted to write my posts in Markdown. Being a rather active Stack Overflow contributor, I became fond of and fluent in it.