Vogone Developer

Vogone Developer
This version won't integrate to the release line, call in the build engineers.

Sunday, December 14, 2014

À la carte Perforce in the DMZ.

How I get access to my main Perforce depot with out shell access on the Perforce server or allowing an inbound firewall policy to the LAN.  The following diagram describes the use of ssh port forwarding with the Perforce p4broker on unprivileged ports by a unprivileged user.


  1. Generate a p4broker config file
    p4broker -C >>  p4broker.conf
  2. Modify the p4broker.conf target and listen ports and protocols as needed. 

    target  = ssl:perforce.server.1666.com:1667;
    listen  = localhost:1666;  # This could be a SSL Port if wanted.

    This will make an SSL based connection between the proxy host and the Perforce server, Then present a unencrypted port to only local users of the proxy server.
    As that we will be using SSH port forwarding to present are p4broker port to the DMZ host.
    I don't necessarily need to run the p4broker listen port with the SSL protocol.
  3. Initialize a ssh reverse port  forward connect from the LAN.
    This will make available port 1666 to the DMZ host web.application.com on localhost.

    devguy@ssh.p4broker.com:~$ ssh -R 1666:localhost:1666 devguy@web.application.com

    Now that we have connected to our DMZ host: 
    devguy@web.application.com~$ netstat -an  |grep 1666
    tcp        0      0 127.0.0.1:1666          0.0.0.0:*               LISTEN

    Lets see if we can get the port forward to work with a p4 info and a list of Perforce users.
    devguy@web.application.com~$ p4 -p 127.0.0.1:1666 users
    devguy <devguy@web.application.com> (devguy) accessed 2014/12/14

    devguy@web.application.com~$ p4 -p 127.0.0.1:1666 info
    User name: devguy
    Client name: web.application.com-cfg
    Client host: web.application.com
    Client unknown.
    Current directory: /home/devguy
    Peer address: 192.168.2.100:51708
    Client address: 127.0.0.1
    Server address: ssl:perforce.server.1666.com:1667
    Server root: /opt/perforce/servers/1666/root
    Server date: 2014/12/14 18:05:30 -0800 PST
    Server uptime: 02:53:33
    Server version: P4D/LINUX26X86_64/2014.2/962050 (2014/11/13)
    Broker address: localhost:1666
    Broker version: P4BROKER/LINUX26X86_64//2014.2/962050
    Server license: none
    Case Handling: sensitive
     
  4. SSH and Perforce login session management is important so log off when your done.
    If you leave idle SSH connections hanging around this solution is not much better then a direct connection to the Perforce server.  My Perforce accounts used in the DMZ tier have short login sessions and my p4broker config is paired with Perforce protections to properly gate which users can access which code lines and what they can do with them. The p4broker offers command filtering and other advance features. Your SSH server can be configure the have an aggressive idle detection which will drop the port forward to your broker after a set amount of time.     

Sunday, December 7, 2014

Perforce and SaltStack Orchestration

Perforce Administrative Tips & Tricks

Quick and dirty simplified versioning of files in mass with SaltStack and Perforce.

I have a dedicated Perforce depot used for configuration management.

My basic depot layout:   

        //sa/...                       #sa for sysadmin
       //sa/cfg/...                  #Configuration directory  for my global template and config files.
       //sa/dev/webapp/...    #My software under development
       //sa/hosts/...               #A hosts directory to version files from hosts.
   
I typically map my hosts out by fqdn. 
       //sa/hosts/{FQDN}/...
       example: 
       //sa/hosts/flagship.vogon.com/...
       //sa/hosts/poetry.vogon.com/...
       //sa/hosts/airlock.vogon.com/...

In Perforce the FQDN mapping to a depot path can be achieved by uses of the perforce client.  

Client configuration tips configuration management:
  1. Come up with a naming convention for your perforce clients:
     My convention is as follows hostname-{postfix} or appstack-{postfix}
    example: p4 -c airlock-cfg
  2. Map the root of your filesystem to a desired depot path:
    I want root "/" on unix host airlock to map //sa/hosts/airlock.vogon.com/...

    Lets have a look at the client spec:
    p4 client -o airlock-cfg

    Client:    airlock-cfg

    Root:    /

    View:
        //sa/hosts/airlock.vogon.com/... //airlock-cfg/...
  3. The use of the hostname or the FQDN as the client spec name with a pre or postfix is very useful, then you want to auto pair the client to the system you want to version.
 Lets run a bunch of simple Perforce versioning commands via Salt cmd.run on a set of hosts
  1. We need a universal Perforce login ticket that can be used across a set of hosts.
    Print a ticket that works on all hosts:
    p4 -u cfg-user login -a -p
    Enter password:
    854B5EA3FB600EF562DFT6Y653C79
  2. Lets add, edit and submit some files to a class of hosts on the vogon.com domain.
    From your salt-master host with already setup salt minions.
    We will orchestrate the versioning of files on the salt minions from the salt master.
    By running the commands remotely via salt's cmd.run

    example:
    root@salt-master.vogon.com:#    salt 'dns*.vogon.com'  cmd.run 'p4 -p poetry.vogon.com:4242 -u cfg-user -P 854...9 -c `hostname`-cfg add /etc/bind/named.conf.options'

    This command will mark for add the named.conf.options file on all hosts that's start with the hostname of dns on the vogon.com domain.

    As that we named the Perforce client spec with the systems hostname we can auto generate the hostname  on the fly as we version. note the backtick surrounding the `hostname` command.
    This will perform the inline printing of the hostname.  
    This should produce a client spec with a name that matches are naming convention.

    Salt cmd.run should run the following Perforce commands correctly on all dns servers.
    p4 -p poetry.vogon.com:4242 -u cfg-user -P 854...9 -c dnsmaster-cfg add /etc/bind/named.conf.options
    p4 -p poetry.vogon.com:4242 -u cfg-user -P 854...9 -c dnsslave-us-cfg add /etc/bind/named.conf.options
    p4 -p poetry.vogon.com:4242 -u cfg-user -P 854...9 -c dnsslave-jp-cfg add /etc/bind/named.conf.options
    p4 -p poetry.vogon.com:4242 -u cfg-user -P 854...9 -c dnsslave-eu-cfg add /etc/bind/named.conf.options

    Your expected Salt and Perforce output should look like this:
    dnsmaster.vogon.com:
        //sa/hosts/dnsmaster.vogon.com/etc/bind/named.conf.options#1 - opened for add
    dnsslave-us.vogon.com:
        //sa/hosts/dnsslave-us.vogon.com/etc/bind/named.conf.options#1 - opened for add
    dnsslave-jp.vogon.com:
        //sa/hosts/dnsslave-jp.vogon.com/etc/bind/named.conf.options#1 - opened for add
    dnsslave-eu.vogon.com:
        //sa/hosts/dnsslave-eu.vogon.com/etc/bind/named.conf.options#1 - opened for add
  3. Lets edit our files in mass from all salt minions.
    example:
    root@salt-master.vogon.com:#    salt 'dns*.vogon.com'  cmd.run 'p4 -p poetry.vogon.com:4242 -u cfg-user -P 854...9 -c `hostname`-cfg edit /etc/bind/named.conf.options'

    Your expected Salt and Perforce output should look like this:
    dnsmaster.vogon.com:
        //sa/hosts/dnsmaster.vogon.com/etc/bind/named.conf.options#1 - opened for edit
    dnsslave-us.vogon.com:
        //sa/hosts/dnsslave-us.vogon.com/etc/bind/named.conf.options#1 - opened for edit
    dnsslave-jp.vogon.com:
        //sa/hosts/dnsslave-jp.vogon.com/etc/bind/named.conf.options#1 - opened for edit
    dnsslave-eu.vogon.com:
        //sa/hosts/dnsslave-eu.vogon.com/etc/bind/named.conf.options#1 - opened for edit
  4.   Lets added some comments to our config files.
    salt 'dns*.vogon.com'  cmd.run 'echo "// This file is versioned" >> /etc/bind/named.conf.options''
  5. Lets submit our files in mass from all salt minions.
    salt 'dns*.vogon.com'  cmd.run 'p4 -p poetry.vogon.com:4242 -u cfg-user -P 854...9 -c `hostname`-cfg submit -d "mass submit of bind9 config files."'

    dnsmaster.vogon.com:
        Submitting change 239725.
        Locking 1 files ...
        add //sa/hosts/dnsmaster.vogon.com/etc/bind/named.conf.optionn#1   
        Change 239725 submitted.
    dnsslave-us.vogon.com:
        Submitting change 239726.
        Locking 1 files ...
        add //sa/hosts/dnsslave-us.vogon.com/etc/bind/named.conf.options#1
        Change 239726 submitted.
    dnsslave-eu.vogon.com:
        Submitting change 239727.
        Locking 1 files ...
        add //sa/hosts/dnsslave-us.vogon.com/etc/bind/named.conf.options#1
        Change 239727 submitted.
    dnsslave-eu.vogon.com:
        Submitting change 239727.
        Locking 1 files ...
        add //sa/hosts/dnsslave-eu.vogon.com/etc/bind/named.conf.options#1
        Change 239727 submitted.
  6. Lets sync to deploy one of our application stacks for DNS in mass from all salt minions.
    example:
    root@salt-master.vogon.com:#    salt 'dns*.vogon.com'  cmd.run 'p4 -p poetry.vogon.com:4242 -u cfg-user -P 854...9 -c dns_webapp-cfg sync //sa/dev/webapp/... '
  7. Lets logout and invalidate the ticket we used across the set of hosts.
    example:
    p4 -p poetry.vogon.com:4242 -u cfg-user logout -a

Friday, November 14, 2014

Jira Administration notes

November 14/14
  • Find a way to change the account ownership of over 500 dashboards and filters in Jira.
   
Potential Solution - https://jamieechlin.atlassian.net/wiki/display/GRV/Script+Runner

Yeap, looks like the Groovy Script Runner plugin will do the job.
https://marketplace.atlassian.com/plugins/com.onresolve.jira.groovy.groovyrunner