Steffen GebertJekyll2024-02-13T22:01:48+01:00//st-g.de/Steffen Gebert//st-g.de/st+blog-spam@st-g.de
//st-g.de/2023/10/grafana-plugin-debugging-on-apple-silicon
//st-g.de/2023/10/grafana-plugin-debugging-on-apple-silicon2023-10-31T23:00:00+01:002023-10-31T23:00:00+01:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>I was recently working on my second Grafana plugin, this time containing a backend plugin. In that process, I was running into quite some challenges - mostly caused by myself not being a very good Go developer.</p>
<p>As I disliked debugging via logging to stdout already in good old PHP times (where the time to compile was zero), I wanted to have a proper debugger (apparently <a href="https://github.com/go-delve/delve">Delve</a>) connected via IntelliJ. If you use Goland, I expect everything to be exactly the same.</p>
<p>While <a href="https://grafana.com/developers/plugin-tools/">Grafana’s <code class="language-plaintext highlighter-rouge">create-plugin</code> tool</a> is results in a fantastic developer experience and also, theoretically, generates everything which is needed to attach a debugger, this failed for me on MacOS using Apple Silicon (ARM64, darwinARM64).</p>
<p>If you don’t want to read through my lengthy monologue of things that did <em>not</em> work, you can directly jump to the section <a href="#worked-for-me-building-grafana">Worked for me</a> - or to <a href="#update-what-probably-also-works-homebrew">What probably also works (and might be easier)</a>.</p>
<h2 id="did-not-work-debugging-in-docker">Did not work: Debugging in Docker</h2>
<p>The plugin scaffolding already generates a <code class="language-plaintext highlighter-rouge">docker-compose.yaml</code> file, which will be used when calling <a href="https://grafana.com/developers/plugin-tools/#step-3-run-your-plugin-in-docker"><code class="language-plaintext highlighter-rouge">npm run server</code></a>. This was very comfortable while <strike>trial and error'ing</strike> the initial development phase.</p>
<p>How to run Delve in Docker for Grafana plugin development is described in <a href="https://community.grafana.com/t/how-to-debug-a-backend-plugin-using-docker/63503">this post</a>. For this to work, one would first build the plugin with debug symbols (<code class="language-plaintext highlighter-rouge">mage build:debug</code>) and then kill the <code class="language-plaintext highlighter-rouge">gpx_my_plugin_name_linux_amd64</code> process, after which Grafana would restart the plugin.</p>
<p>Unfortunately, attaching the debugger failed for me with the following error message:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>root@81c9000cfe18:/plugin# dlv attach <span class="nt">--headless</span> <span class="nt">--listen</span><span class="o">=</span>:2345 PID
API server listening at: <span class="o">[</span>::]:2345
2023-10-31T19:51:03Z warning <span class="nv">layer</span><span class="o">=</span>rpc Listening <span class="k">for </span>remote connections <span class="o">(</span>connections are not authenticated nor encrypted<span class="o">)</span>
Warning: no debug info found, some functionality will be missing such as stack traces and variable evaluation.
could not attach to pid 3511: could not <span class="nb">read </span>debug info <span class="o">(</span>decoding dwarf section info at offset 0x0: too short<span class="o">)</span> and could not <span class="nb">read </span>go symbol table <span class="o">(</span>could not <span class="nb">read </span>section .gopclntab<span class="o">)</span>
</code></pre></div></div>
<p>I’m not sure, where this is coming from. I have the assumption it is a limitation of the tool I use to run containers on my Mac (<a href="https://orbstack.dev/">OrbStack</a>), which is in general pretty cool, but I also ran into problems playing with eBPF programs. So it might be that despite having capability <code class="language-plaintext highlighter-rouge">SYS_PTRACE</code>, the <code class="language-plaintext highlighter-rouge">dlv</code> is not allowed to trace the process. If you use another tool (Docker Desktop, Rancher, etc.), maybe it works for you.</p>
<h2 id="did-not-work-running-grafana-on-macos">Did not work: Running Grafana on MacOS</h2>
<p>As next step, I downloaded Grafana for Mac from their <a href="https://grafana.com/grafana/download?pg=get&platform=mac&plcmt=selfmanaged-box1-cta1&edition=oss">download page</a>. It can be started using <code class="language-plaintext highlighter-rouge">bin/grafana server</code>.</p>
<p>After symlinking my plugin’s <code class="language-plaintext highlighter-rouge">dist/</code> folder into a subfolder of <code class="language-plaintext highlighter-rouge">data/plugins/</code> (don’t put the <code class="language-plaintext highlighter-rouge">gpx_*</code> files directly into <code class="language-plaintext highlighter-rouge">data/plugins/</code>, your plugin will not be discovered), I was happy to see my plugin being loaded.</p>
<p>However, at the time of writing (Grafana version 10.2.0), there is still no ARM64 build available. While this appears reasonable (who wants to run Grafana on a Mac!), this makes it complicated for developing on a Mac. Luckily, around that time, I discovered that parts of what had to be done manually before are actually covered in a target</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>mage debugger
</code></pre></div></div>
<p>This step is inherited from <a href="https://github.com/grafana/grafana-plugin-sdk-go/blob/8140bf11e904522b6c91020d2efb12a6b62d6957/build/common_unix.go#L48-L105">grafana-plugin-sdk-go</a>.</p>
<p>Other available targets are:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>mage <span class="nt">-l</span>
Targets:
build:backend build a production build <span class="k">for </span>the current platform
build:darwin builds the back-end plugin <span class="k">for </span>OSX.
build:darwinARM64 builds the back-end plugin <span class="k">for </span>OSX on ARM <span class="o">(</span>M1<span class="o">)</span><span class="nb">.</span>
build:debug builds the debug version <span class="k">for </span>the current platform
build:generateManifestFile generates a manifest file <span class="k">for </span>plugin submissions
build:linux builds the back-end plugin <span class="k">for </span>Linux.
build:linuxARM builds the back-end plugin <span class="k">for </span>Linux on ARM.
build:linuxARM64 builds the back-end plugin <span class="k">for </span>Linux on ARM64.
build:windows builds the back-end plugin <span class="k">for </span>Windows.
buildAll<span class="k">*</span> builds production executables <span class="k">for </span>all supported platforms.
clean cleans build artifacts, by deleting the dist directory.
coverage runs backend tests and makes a coverage report.
debugger makes a new debug build, re-launches the plugin and attaches the Delve debugger, <span class="k">in </span>headless mode listening on port 3222.
e2E:append starts the E2E proxy <span class="k">in </span>append mode.
e2E:certificate prints the CA certificate to stdout.
e2E:overwrite starts the E2E proxy <span class="k">in </span>overwrite mode.
e2E:replay starts the E2E proxy <span class="k">in </span>replay mode.
format formats the sources.
lint audits the <span class="nb">source </span>style
reloadPlugin - kills any running instances and waits <span class="k">for </span>grafana to reload the plugin
<span class="nb">test </span>runs backend tests.
watch rebuilds the plugin backend when files change.
<span class="k">*</span> default target
</code></pre></div></div>
<p>However, having Grafana built for Intel CPUs (AMD64) running, this also loaded the <code class="language-plaintext highlighter-rouge">darwin_amd64</code> build of the plugin. In contrast, the <code class="language-plaintext highlighter-rouge">make build:debug</code> (which is automatically invoked by the <code class="language-plaintext highlighter-rouge">debugger</code> target) is only creating a debug build for the <em>current</em> platform. My Mac’s current platform is ARM64, not AMD64!</p>
<p>While I’m sure there is a way to fix this, I moved on to whatever next came to my mind…</p>
<h1 id="worked-for-me-building-grafana">Worked for me: Building Grafana</h1>
<p>While no download was available, I was optimistic to get Grafana built myself. As usual for the Grafana community, building it is very well described in the <a href="https://github.com/grafana/grafana/blob/HEAD/contribute/developer-guide.md#build-grafana">Developer Guide</a>. After building the frontend with YARN, the server can be started using</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ make run
</code></pre></div></div>
<p>You can now see in the <em>Activity Monitor</em> that <code class="language-plaintext highlighter-rouge">grafana</code> runs as Kind <code class="language-plaintext highlighter-rouge">Apple</code>, where in contrast it was before <code class="language-plaintext highlighter-rouge">Intel</code>:</p>
<p><img src="/images/2023-10-31-grafana-plugin-debugging/grafana-activity-monitor.png" alt="Activity Monitor screenshot showing the grafana process running as Kind Apple" /></p>
<p>To allow unsigned plugins to be executed, follow the documentation and update <code class="language-plaintext highlighter-rouge">conf/defaults.ini</code> to</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>app_mode = development
</code></pre></div></div>
<p>After installing my plugin in Grafana, it is also shown as <code class="language-plaintext highlighter-rouge">Apple</code> and also with the <code class="language-plaintext highlighter-rouge">*_darwin_arm64</code> file name.</p>
<p><img src="/images/2023-10-31-grafana-plugin-debugging/grafana-activity-monitor-plugins.png" alt="Activity Monitor screenshot showing the grafana process running as Kind Apple" /></p>
<p>If the plugin is not shown in Grafana, make sure the folder structure is correct, which you can easily check by installing another plugin for comparison.</p>
<p>While you have Grafana running (via above <code class="language-plaintext highlighter-rouge">make run</code>), execute in your plugin’s folder:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>mage debugger
</code></pre></div></div>
<p>You will first see Grafana complaining that the plugin process exited (it will immediately restart it):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ERROR[10-31|21:31:00] plugin process exited logger=plugin.foobar-something-datasource path=/Users/.../foobar-something-datasource/dist/gpx_some_thing_darwin_arm64 pid=23337 error="signal: killed"
</code></pre></div></div>
<p>while your other console will (hopefully!) report success:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mage debugger
API server listening at: <span class="o">[</span>::]:3222
2023-10-31T21:31:01+01:00 warning <span class="nv">layer</span><span class="o">=</span>rpc Listening <span class="k">for </span>remote connections <span class="o">(</span>connections are not authenticated nor encrypted<span class="o">)</span>
2023-10-31T21:31:01+01:00 info <span class="nv">layer</span><span class="o">=</span>debugger attaching to pid 23517
2023-10-31T21:31:01+01:00 debug <span class="nv">layer</span><span class="o">=</span>debugger entryPoint 0x1044bc000 machoOff 0x100000000
2023-10-31T21:31:01+01:00 warning <span class="nv">layer</span><span class="o">=</span>debugger debug_frame workaround not applied: <span class="k">function </span>internal/abi.<span class="o">(</span><span class="k">*</span>RegArgs<span class="o">)</span>.Dump <span class="o">(</span>at 0x1044bd070<span class="o">)</span> covered by 0x1044bd070-0x1044bd240
2023-10-31T21:31:01+01:00 debug <span class="nv">layer</span><span class="o">=</span>debugger Adding target 23517 <span class="s2">""</span>
</code></pre></div></div>
<p>Now, we need to connect IntellJ / Goland to the remote debugger. If you use VSCode as IDE, then please consult the Internet on how to connect to <code class="language-plaintext highlighter-rouge">dlv</code>, as the remaining steps will be different.</p>
<p>Under <code class="language-plaintext highlighter-rouge">Run</code> > <code class="language-plaintext highlighter-rouge">Edit Configurations...</code>, add a new <code class="language-plaintext highlighter-rouge">Go Remote</code> configuration for port 3222:</p>
<p><img src="/images/2023-10-31-grafana-plugin-debugging/intellij-debug-config.png" alt="IntelliJ Edit Configurations screen" />.</p>
<p>Pick a line in your code and place a breakpoint and start the debugger using the bug icon on the upper right.</p>
<p><img src="/images/2023-10-31-grafana-plugin-debugging/debugger-attached.png" alt="IntelliJ Edit Configurations screen" />.</p>
<p>Depending on where in your code you placed the breakpoint, you might have to execute a query.</p>
<p>I hope this helps you with developing your plugin. In case you have any corrections to this article, you’re welcome to contact me via <code class="language-plaintext highlighter-rouge">@StGebert@digitalcourage.social</code>.</p>
<p>Happy debugging!</p>
<h2 id="update-what-probably-also-works-homebrew">Update: What probably also works: Homebrew</h2>
<p>After posting about this article [in the Grafana community](https://community.grafana.com/t/plugin-debugging-on-macos-apple-silicon/106799/, I got the feedback that installing it via homebrew would also provide an ARM64 Grafana.</p>
<p>Indeed, this is true:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>brew <span class="nb">install </span>grafana
<span class="o">==></span> Downloading https://ghcr.io/v2/homebrew/core/grafana/manifests/10.1.1
<span class="c">#################################################################################################################################################################################################### 100.0%</span>
<span class="o">==></span> Fetching grafana
<span class="o">==></span> Downloading https://ghcr.io/v2/homebrew/core/grafana/blobs/sha256:31f547ed00edb1ac7b60a764fee642f70c1eecb811400974f266861a8efba094
<span class="c">#################################################################################################################################################################################################### 100.0%</span>
<span class="o">==></span> Pouring grafana--10.1.1.arm64_ventura.bottle.tar.gz
<span class="o">==></span> Caveats
To start grafana now and restart at login:
brew services start grafana
Or, <span class="k">if </span>you don<span class="s1">'t want/need a background service you can just run:
/opt/homebrew/opt/grafana/bin/grafana server --config /opt/homebrew/etc/grafana/grafana.ini --homepath /opt/homebrew/opt/grafana/share/grafana --packaging\=brew cfg:default.paths.logs\=/opt/homebrew/var/log/grafana cfg:default.paths.data\=/opt/homebrew/var/lib/grafana cfg:default.paths.plugins\=/opt/homebrew/var/lib/grafana/plugins
</span></code></pre></div></div>
<p>Especially the last command is nice, because I usually struggle to find around where homebrew has put the config files.</p>
<p><a href="//st-g.de/2023/10/grafana-plugin-debugging-on-apple-silicon">Grafana Plugin Debugging on Apple Silicon</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on October 31, 2023.</p>
//st-g.de/2023/02/aws-this-is-my-architecture
//st-g.de/2023/02/aws-this-is-my-architecture2023-02-01T23:00:00+01:002023-02-01T23:00:00+01:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p><a href="https://aws.amazon.com/architecture/this-is-my-architecture">This is My Architecture</a> is a video series produced by AWS, where customers are presenting interesting architectures through short video clips.</p>
<p>I had the pleasure and honor to present emnify’s packet gateway architecture in such a video. You can see the result here:</p>
<p><a href="https://www.youtube.com/watch?v=SC6n6J8Bi58">EMnify: Building a Cloud Native Mobile Network for IoT Leveraging AWS’s Global Infrastructure</a></p>
<iframe width="420" height="315" src="http://www.youtube.com/embed/SC6n6J8Bi58" frameborder="0" allowfullscreen=""></iframe>
<p>Thanks to our amazing account manager Karl Oppermann, the AWS TMA team, as well as Amazon’s video crew came over to Munich.
In a rented film studio, the crew recorded ca. 20 customer videos. It was a pleasure, to be one of those customers.
My host in this video was Thomas Wieger, a Senior Solutions Architect at AWS.</p>
<p><img src="/images/2023-02-01-aws-this-is-my-architecture/steffen-and-thomas.jpg" alt="Steffen and Thomas" />
<img src="/images/2023-02-01-aws-this-is-my-architecture/steffen-eisbach-studios.jpg" alt="Steffen in front of the Eisbach film studios" /></p>
<p><a href="//st-g.de/2023/02/aws-this-is-my-architecture">AWS This is My Architecture Video</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on February 01, 2023.</p>
//st-g.de/2020/12/interview-emnify-blog
//st-g.de/2020/12/interview-emnify-blog2020-12-11T23:00:00+01:002020-12-11T23:00:00+01:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>As I recently gave two talks at the <em>AWS Community Day - Bay Area</em> as well as <em>DENOG12</em>, where I explained how we implemented two of our recent projects at EMnify,
I was interviewed in the <a href="https://www.youtube.com/watch?v=WAypNBSGQpw">EMnify blog</a>.</p>
<p>If you are interested in my other talks, see the <a href="/speaking.md">Speaking</a> page. Happy watching!</p>
<p><a href="//st-g.de/2020/12/interview-emnify-blog">Interview in the EMnify Blog</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on December 11, 2020.</p>
//st-g.de/2019/07/be-careful-with-aws-private-api-gateway-endpoints
//st-g.de/2019/07/be-careful-with-aws-private-api-gateway-endpoints2019-07-24T18:00:00+02:002019-07-24T18:00:00+02:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>As many shops out there, we (at <a href="https://www.emnify.com">EMnify</a>) are using AWS Lambda for an increasing number of applications to make our life easier.
The prerequisite to trigger Lambda functions via REST API calls is to deploy an API Gateway into an AWS account.</p>
<p>Historically, API Gateway offered deployment types:</p>
<ul>
<li><em>regional</em>: what I would consider expected - the API is reachable directly via a DNS host name including the AWS region where it is deployed</li>
<li><em>edge-optimized</em>: what feels weird and almost useless - the API Gateway automatically deploys a somewhat hidden CloudFront distribution, which is not further configurable for the user; I would recommend to use <em>regional</em> deployment and set up the CF distribution yourself instead, if the CloudFront CDN should be used.</li>
</ul>
<p>A bit more than one year ago, AWS has launched a third type:</p>
<ul>
<li><a href="https://aws.amazon.com/blogs/compute/introducing-amazon-api-gateway-private-endpoints/"><em>private</em></a>: <strong>what I consider dangerous</strong> - the API is only reachable via a VPC endpoint inside your VPC.</li>
</ul>
<p>This harsh statement might come unexpected, as most readers might be actually fans of VPC endpoints and so am I. But in this case, the setting <em>Enable Private DNS Name</em>, which I would certainly want to activate, <strong>will prevent all access to any regional/public API Gateway in the whole AWS region!</strong></p>
<p>Assuming DNS host names are enabled in the VPC configuration, this setting for the VPC endpoint changes the DNS resolution for API Gateway host names (<code class="language-plaintext highlighter-rouge"><api-id>.execute-api.<region>.amazonaws.com</code>) to return the IP addresses of the VPC endpoint’s ENIs. So far, this is expected and in line with how all other VPC Interface Endpoints work (like <code class="language-plaintext highlighter-rouge">kms.eu-west-1.amazonaws.com</code> then pointing to private IPs). The caveat however is that <strong>any request to an API Gateways in the whole region will</strong> pass through the VPC endpoint because of one of its DNS entries is <code class="language-plaintext highlighter-rouge">*.execute-api.<region>.amazonaws.com</code>, which will very unexpectedly <strong>be rejected with status <code class="language-plaintext highlighter-rouge">403 Forbidden</code></strong>.</p>
<p>To recap: all calls to API Gateways located in the same AWS region accessed from the whole VPC will return status 403. This caused us headaches, as this problem stayed first unnoticed in our infrastructure and - of course - was not detected when testing the private API Gateway.</p>
<p>So what are the alternatives? One can - and in my opinion has to, unless you can ensure that forever nobody will call a public API from this VPC - disable the private DNS name. Then accessing the private API, however, becomes a lot more complicated.</p>
<p>The documentation <a href="https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-private-apis.html">Create a Private API in Amazon API Gateway</a> as of now (July 2019) reads pretty fine, just as like you use the VPC endpoint for accessing the API. But what does this tiny <code class="language-plaintext highlighter-rouge">Host: ...</code> line mean?</p>
<p><img src="/images/2019-07-aws-api-gateway/apigateway-private-api-accessing-api.png" alt="private API Gateway" /></p>
<p>Reading the page three times still kept me confident I’m doing the right thing.</p>
<p>More clear is this FAQ entry: <a href="https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-vpc-connections/">Why can’t I connect to my public API from an API Gateway VPC endpoint?</a>. This clarifies that one of the two following HTTP headers must me set:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Host: <api-id>.execute-api.<region>.amazonaws.com
</code></pre></div></div>
<p>or</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>x-apigw-api-id: <api-id>
</code></pre></div></div>
<p>when sending the HTTP request to the hostname of the VPC endpoint (<code class="language-plaintext highlighter-rouge">https://<vpce-id>.execute-api.<region>.vpce.amazonaws.com/stageName</code>).
This requires you to have pretty deep control over the applications accessing the private API. And you need to be willing to modify them accordingly.</p>
<p>Another alternative, as I was told by AWS Community Hero <a href="https://www.taimos.de/">Thorsten Höger</a>, which would not require to set custom headers with disabled DNS resolution, would be based on a custom domain name configured in the API Gateway and an ALB in front of it. Based on an IP target group, the ALB would forwards the requests to the ENIs of the VPC Endpoint. Looks like an option to move the complexity into the network.</p>
<p>So far, it became not at all clear to me (and AWS support was not able to explain), why all calls to public APIs are rejected. If this would work the execute-api VPC endpoint would behave like any other - and not kill your setup. #awswishlist</p>
<p>If my understanding is wrong or if you have a better solution, I’m happy to read from you in the comments below!</p>
<p><a href="//st-g.de/2019/07/be-careful-with-aws-private-api-gateway-endpoints">Be careful with AWS Private API Gateway Endpoints</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on July 24, 2019.</p>
//st-g.de/2016/12/parametrized-jenkins-pipelines
//st-g.de/2016/12/parametrized-jenkins-pipelines2016-12-08T17:00:00+01:002016-12-08T17:00:00+01:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>During and after the presentation about Jenkins pipelines that I gave yesterday at the <a href="https://www.meetup.com/de-DE/devops-stuttgart/events/235619665/">DevOps Meetup Stuttgart</a>, we had a couple of very interesting discussions.
One of them was a way to make a Jenkins pipeline job parametrized. Use cases for this that were discussed included different target environments, into which the pipeline should deploy to.</p>
<p>While it is not very obvious, there is one particular step that allows to configure a pipeline job’s properties, including triggers, how to rotate logs, and said input parameters: The <a href="https://jenkins.io/doc/pipeline/steps/workflow-multibranch/#code-properties-code-set-job-properties"><code class="language-plaintext highlighter-rouge">properties</code></a> step contained in the <em>multibranch</em> plugin.
Please don’t expect too much from the referenced documentation, as it is pretty useless.</p>
<p>Of a lot greater help is the Snippet Editor, which is available in every Jenkins instance.</p>
<p><img src="/images/2016-12-08-parametrized-jenkins-pipelines/snippet-editor.png" alt="Snippet Editor" /></p>
<p>Selecting the <code class="language-plaintext highlighter-rouge">properties</code> step from the dropdown menu offers a whole lot of options to select, including <em>This project is parameterized</em>.
After checking this option, parameters of different kinds can be added, i.e., boolean, string, select parameters etc.</p>
<p><img src="/images/2016-12-08-parametrized-jenkins-pipelines/string-parameter.png" alt="Adding a string parameter" /></p>
<p>Hitting the <em>Generate Pipeline Script</em> button emits the following code snippet (manually formatted by myself):</p>
<figure class="highlight"><pre><code class="language-groovy" data-lang="groovy"><span class="n">properties</span><span class="o">([</span>
<span class="n">parameters</span><span class="o">([</span>
<span class="n">string</span><span class="o">(</span><span class="nl">name:</span> <span class="s1">'DEPLOY_ENV'</span><span class="o">,</span> <span class="nl">defaultValue:</span> <span class="s1">'TESTING'</span><span class="o">,</span> <span class="nl">description:</span> <span class="s1">'The target environment'</span><span class="o">,</span> <span class="o">)</span>
<span class="o">])</span>
<span class="o">])</span></code></pre></figure>
<p>Once this code is included at the top level of the pipeline script, any pipeline execution resets the job’s parameters to the specified values.
From that point on, the <em>Build Now</em> button has changed to a <em>Build with Parameter</em> and every time the pipeline is launched, the user is asked to specify defined values.</p>
<p><img src="/images/2016-12-08-parametrized-jenkins-pipelines/build-with-parameters.png" alt="Build with parameters" /></p>
<p>P.S: The first run will most likely fail (<em>EDIT</em>: see last paragraph), as chances are good that you try to access the just added parameters (<a href="https://issues.jenkins-ci.org/browse/JENKINS-40235">JENKINS-40235</a>).
Subsequent runs will work, if your pipeline script is correct.</p>
<h2 id="bonus-selection-from-dropdown">Bonus: Selection from Dropdown</h2>
<p>Using the <em>Choice Parameter</em>, the possible inputs can be restricted to a pre-defined list of choices.</p>
<p><img src="/images/2016-12-08-parametrized-jenkins-pipelines/choice-parameter.png" alt="Adding a choice parameter" /></p>
<p>Be warned that the emitted code including <code class="language-plaintext highlighter-rouge">choices: ['TESTING', 'STAGING', 'PRODUCTION']</code> fails with an exception</p>
<blockquote>
java.lang.ClassCastException: hudson.model.ChoiceParameterDefinition.choices expects class java.lang.String but received class java.util.ArrayList
</blockquote>
<p>Instead, the list of choices has to be supplied as String containing new line characters (<code class="language-plaintext highlighter-rouge">\n</code>): <code class="language-plaintext highlighter-rouge">choices: ['TESTING\nSTAGING\nPRODUCTION']</code> (<a href="https://issues.jenkins-ci.org/browse/JENKINS-40358">JENKINS-40358</a>).</p>
<p><strong>EDIT January, 12th:</strong> Previously, it was common to access these parameters using Groovy or environment variables:</p>
<figure class="highlight"><pre><code class="language-groovy" data-lang="groovy"><span class="n">echo</span> <span class="s2">"Will deploy to ${DEPLOY_ENV}"</span></code></pre></figure>
<p>As of <a href="https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Groovy+Plugin#PipelineGroovyPlugin-2.18%28Sep23%2C2016%29"><code class="language-plaintext highlighter-rouge">workflow-cps</code> version 2.18</a>, a new <code class="language-plaintext highlighter-rouge">params</code> global variable provides sane access also on the first run (by returning specified default values).</p>
<figure class="highlight"><pre><code class="language-groovy" data-lang="groovy"><span class="n">echo</span> <span class="s2">"Will deploy to ${params.DEPLOY_ENV}"</span></code></pre></figure>
<p><a href="//st-g.de/2016/12/parametrized-jenkins-pipelines">Parametrized Jenkins Pipelines</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on December 08, 2016.</p>
//st-g.de/2016/11/first-devops-meetup-wuerzburg
//st-g.de/2016/11/first-devops-meetup-wuerzburg2016-11-08T07:30:00+01:002016-11-08T07:30:00+01:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>As a long-time visitor of the <a href="https://www.meetup.com/de-DE/DevOps-Frankfurt/">DevOps Frankfurt</a> meetup, I always had the desire for establishing the same kind of event here in my beautiful town of <a href="https://www.google.de/maps/place/Würzburg/">Würzburg</a>.
Together with Andreas Rudat (<a href="https://www.mayflower.de">mayflower</a>) and Sebastian Kremer (<a href="http://www.eikona.de">Eikona</a>), we finally made it happen with our first event of <a href="https://www.meetup.com/de-DE/DevOps-Wuerzburg-Mainfranken/events/234778486/">DevOps Würzburg Mainfranken</a>.</p>
<p>The diversity between the in total 40 participants lead to very broad discussions, with a lot of thoughts focused on DevOps culture and why we need it.
Participants ranged from roughly a third of them being students (some even in their first semester) to CEO, from many in the role of developer, over sysadmins to project managers and Technical Evangelist.</p>
<p>To get people familiar with the ideas and thinking of the DevOps movement and to make them clear that this is not a set of tooling that solves one’s day-to-day problems immediately, we prepared two introductory talks.</p>
<ul>
<li>Lorenz Weber (mayflower) talked about the questions that DevOps answers and the new ones that appear out of that - and if there is this kind of a “DevOp”.</li>
<li>I myself gave an <a href="http://www.slideshare.net/StephenKing/continuous-delivery-68335663">introduction to the concept of Continuous Delivery</a>, which is strongly interwoven with Agile development and builds opon the DevOps thinking to deliver software faster and better.</li>
</ul>
<p>To summarize this evening, I was very happy to see that many participants - given the size of our “village” and compared to my experience with other meetups. Probably, the location in the computer science building at the University of Würzburg motivated more students, is easy to reach, and also a “neutral” place.</p>
<p>The only aspect that is disappointing to me is that not a single one of the other (~80) PhD candidates and researchers here in the computer science faculty joined us. This reminds me once more about my feelings that I am wrong in academia and have to get out here ASAP (luckily, this will really happen very soon).</p>
<p>As the spirit of the participants definitely motivated us to continue with this meetup series, we already agreed on the next talk being once again focused on cultural aspects. <a href="https://twitter.com/seklenk">Sebastian Klenk</a>, Technical Evangelist at Microsoft, will talk about the experience and transformations happening in his organization. The date will be announced soon via the <a href="https://www.meetup.com/de-DE/DevOps-Wuerzburg-Mainfranken/">meetup group</a>.</p>
<p>Finally, I want to thank my friend Andy Grunwald for <a href="http://andygrunwald.com/blog/lesson-learned-from-running-a-local-meetup/">sharing his lessens learned</a> from running the <a href="https://www.meetup.com/de-DE/Web-Engineering-Duesseldorf/">Web Engineering Meetup Düsseldorf</a> (former PHP User Group) which helped me and us to prepare this event.</p>
<p><a href="//st-g.de/2016/11/first-devops-meetup-wuerzburg">First DevOps Meetup in Würzburg</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on November 08, 2016.</p>
//st-g.de/2016/10/devopscamp-nuernberg
//st-g.de/2016/10/devopscamp-nuernberg2016-10-08T20:30:00+02:002016-10-08T20:30:00+02:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>Right now, I am sitting in the train on my way back from the <a href="http://openspacer.org/60-devops-community/127-devops-camp-compact-2016/">DevOps Camp Compact</a> in Nürnberg. It was a very exhausting day with two talks that I gave and a lot of information and energy that I got from it.
I want to thank the organizers Tobias and Stefan, the location sponsor (my TYPO3 friends Netlogix), and the roughly 100 participants that made this such a great day. Further, I’m happy that I was finally able to join the 6th edition after I wasn’t able to attend the previous four events.</p>
<p>During the session planning, I offered three talks about the following topics:</p>
<ul>
<li>HTTP/2, QUIC, and Multipath-TCP</li>
<li>Jenkins pipelines</li>
<li>SDN & NFV from the academic perspective</li>
</ul>
<p>Based on the interest of the participants, I finally held sessions about the first two topics.</p>
<h2 id="sessions-held-by-me">Sessions Held by Me</h2>
<h3 id="http2-quic-and-multipath-tcp">HTTP/2, QUIC, and Multipath-TCP</h3>
<p>This talk brought in some of my network expertise that I gained from my $dayjob at the university. I started with the history of HTTP, the protocol, which powers the web. Regarding innovative protocols, I first covered HTTP/2, a protocol that should be used by everybody right now. The next one, QUIC (Quick UDP Internet Connections), is currently used by Google and they are using us as their testers (if we use Chrome and access Google services). In contrast to HTTP, it builds up on UDP instead of TCP. Finally, I covered Multipath TCP, which allows a multi-homed device to transfer data simultaneously via multiple connections. The unfortunate state is that the only public deployment is Apple Siri, which uses it only for reliability, but not for load balancing.</p>
<p>The slides are publicly available on <a href="http://www.slideshare.net/StephenKing/cleaning-up-the-dirt-of-the-nineties-how-new-protocols-are-modernizing-the-web">SlideShare</a>. To save some disk space, I didn’t duplicate them, but refer to the identical slide deck that I used during TYPO3camp Vienna.</p>
<h3 id="jenkins-pipelines">Jenkins Pipelines</h3>
<p>Very interesting was also my second talk – for me, and I hope also for the rest of the audience. As there were a couple of people in the room, who have way more experience with Jenkins than I have, we had some very good discussions.</p>
<p>In short: The Jenkins pipeline plugins provide a modern way to describe delivery pipelines as code, much cleaner than previously possible with Jenkins.</p>
<p>The slides for this talk are available via <a href="https://github.com/StephenKing/dvocc16-jenkins-pipeline">GitHub Pages</a>.</p>
<p>The new things, which I learned include:</p>
<ul>
<li>The <a href="https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Milestone+Step+Plugin">Pipeline Milestone Step Plugin</a> allows to automatically cancel older pipeline instances, once a newer one reached a certain (<code class="language-plaintext highlighter-rouge">milestone</code>) step. This is particular useful, if a manual sign-off step is not acknowledged and it is (as usually) not desired that old pipeline executions remain running, or when (who knows why) a newer commit passes faster through a deployment pipeline than an old one, which could result in the old state being deployed over the newer.</li>
<li>The JobDSL plugin is still useful in sitations, where folks are not working with GitHub or Bitbucket and could be used to generate the pipeline jobs. (I never said that this plugin isn’t useful, but I wouldn’t have seen it together in relation with pipeline jobs.)</li>
<li>Pipeline versioning could be done using the new <a href="https://github.com/jenkinsci/workflow-cps-global-lib-plugin/blob/master/README.md#"><code class="language-plaintext highlighter-rouge">@Library</code></a> annotation on a per repo level within the <code class="language-plaintext highlighter-rouge">Jenkinsfile</code> by letting it point a specific branch of the pipeline repo.</li>
</ul>
<h2 id="others-sessions">Other’s Sessions</h2>
<p>We had an interesting session during the “Docker Security” session. Some of the key points here are:</p>
<ul>
<li>Comtainers are not VMs. They do not offer the same way of isolation.</li>
<li>Every user, who has access to the Docker socket (<code class="language-plaintext highlighter-rouge">/var/run/docker</code>), is root-equivalent. By mounting in directories and <code class="language-plaintext highlighter-rouge">sudo</code>-ing within the container, any file on the host can be edited.</li>
<li>In some situations, Docker exposes more ports via <code class="language-plaintext highlighter-rouge">iptables</code> than the user might expect. Therefore, it can be helpful to start an additional (privileged) container within the same network namespace of the container to protect and block all traffic to ports, which are not explicitly white-listed.</li>
<li>Cgroups should be used to limit resource usage by single containers.</li>
</ul>
<p>During the “Where should we run those containers?” session, we discussed the cool kids Kubernetes, Mesos/DCOS, and Rancher. Nobody knows the answer, what to use ☺. While couple of people have been experimenting with some of them, none is using any of them in production. I definitely will experiment a bit more about Kubernetes, while I find Mesos/DCOS still extremely interesting.</p>
<h2 id="summary">Summary</h2>
<p>I really enjoyed being there. The people and the discussions with them were great and I hope that I will make it to the next edition somewhen in April/May 2017 again in Nürnberg. If you’re interested, keep an eye on <a href="http://www.devops-camp.de">devops-camp.de</a>.</p>
<p><a href="//st-g.de/2016/10/devopscamp-nuernberg">DevOps Camp Compact 2016</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on October 08, 2016.</p>
//st-g.de/2016/08/jenkins-pipeline-autocompletion-in-intellij
//st-g.de/2016/08/jenkins-pipeline-autocompletion-in-intellij2016-08-12T22:00:00+02:002016-08-12T22:00:00+02:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>I am a big fan of the new <a href="https://jenkins.io/pipeline/getting-started-pipelines/">Jenkins Pipeline</a> suite and enjoy defining my pipelines as code.
Now, I noticed that there is even a way to make IntelliJ IDEA aware ofthe pipeline DSL syntax, which supports the developer with autocompletion and documentation.</p>
<p>For looking up the DSL syntax, I am a frequent visitor of the <a href="https://jenkins.io/doc/pipeline/steps/">Pipeline Steps Reference</a> on <a href="https://jenkins.io">jenkins.io</a>.
As I am only seldom visiting the Jenkins web interface to use the <em>Pipeline Syntax Snippet Generator</em>, I was a bit puzzled when I stumbled over a mysterious <em>IntelliJ IDEA GDSL</em> link (available for logged-in users).</p>
<p><img src="/images/2016-08-12-jenkins-pipeline-autocompletion-in-intellij/GDSL-link.png" alt="Jenkins with link to the IntelliJ IDEA GDSL file" /></p>
<p>After reading these words and curiously following the link, my confidence that this would finally help me (still a Groovy novice) to improve my experience of coding with the <em>roovy DSL used by the pipeline plugin. I already used the _Groovy</em> plugin in IntelliJ IDEA, but, of course, all syntax of the Pipeline DSL were unknown until this point.</p>
<p>The content of the text file shown after following the link started with the following lines:</p>
<figure class="highlight"><pre><code class="language-groovy" data-lang="groovy"><span class="c1">//The global script scope</span>
<span class="kt">def</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">context</span><span class="o">(</span><span class="nl">scope:</span> <span class="n">scriptScope</span><span class="o">())</span>
<span class="n">contributor</span><span class="o">(</span><span class="n">ctx</span><span class="o">)</span> <span class="o">{</span>
<span class="n">method</span><span class="o">(</span><span class="nl">name:</span> <span class="s1">'build'</span><span class="o">,</span> <span class="nl">type:</span> <span class="s1">'Object'</span><span class="o">,</span> <span class="o">...</span>
<span class="n">method</span><span class="o">(</span><span class="nl">name:</span> <span class="s1">'build'</span><span class="o">,</span> <span class="nl">type:</span> <span class="s1">'Object'</span><span class="o">,</span> <span class="o">...</span>
<span class="n">method</span><span class="o">(</span><span class="nl">name:</span> <span class="s1">'echo'</span><span class="o">,</span> <span class="nl">type:</span> <span class="s1">'Object'</span><span class="o">,</span> <span class="nl">params:</span> <span class="o">[</span><span class="nl">message:</span><span class="s1">'java.lang.String'</span><span class="o">],</span> <span class="nl">doc:</span> <span class="s1">'Print Message'</span><span class="o">)</span>
<span class="n">method</span><span class="o">(</span><span class="nl">name:</span> <span class="s1">'error'</span><span class="o">,</span> <span class="nl">type:</span> <span class="s1">'Object'</span><span class="o">,</span> <span class="nl">params:</span> <span class="o">[</span><span class="nl">message:</span><span class="s1">'java.lang.String'</span><span class="o">],</span> <span class="nl">doc:</span> <span class="s1">'Error signal'</span><span class="o">)</span>
<span class="n">method</span><span class="o">(</span><span class="nl">name:</span> <span class="s1">'input'</span><span class="o">,</span> <span class="nl">type:</span> <span class="s1">'Object'</span><span class="o">,</span> <span class="nl">params:</span> <span class="o">[</span><span class="nl">message:</span><span class="s1">'java.lang.String'</span><span class="o">],</span> <span class="nl">doc:</span> <span class="s1">'Wait for interactive input'</span><span class="o">)</span></code></pre></figure>
<p>Yes, the known pipeline steps <code class="language-plaintext highlighter-rouge">build</code>, <code class="language-plaintext highlighter-rouge">echo</code>, <code class="language-plaintext highlighter-rouge">error</code>, etc. That’s what I need to make IntelliJ aware of. But I had no clue, how to get that into the IDE.</p>
<p>Following the <a href="https://confluence.jetbrains.com/display/GRVY/Scripting+IDE+for+DSL+awareness">documentation “Scripting IDE for DSL awareness”</a> and <a href="https://gist.github.com/gclayburg/0107bdeb38349c6fc87a">this Gist</a>, I figured out that I have to place this <em>“somewhere in the classpath”</em>. Nowhere within the project worked out, however.
<a href="https://github.com/TYPO3-infrastructure/jenkins-pipeline-global-library-chefci/">My project</a>, is a <a href="https://github.com/jenkinsci/workflow-cps-global-lib-plugin">Pipeline Global Library</a>, which allows to define functionality used in <code class="language-plaintext highlighter-rouge">Jenkinsfile</code>s of multiple projects. The same, however, should apply to projects that only make use of the <code class="language-plaintext highlighter-rouge">Jenkinsfile</code>.</p>
<p>After putting the contents of the Jenkins output into a <a href="https://github.com/TYPO3-infrastructure/jenkins-pipeline-global-library-chefci/blob/7c4fbc063cd033e36fdc4457d0cac4938e102b70/src/pipeline.gdsl"><code class="language-plaintext highlighter-rouge">pipeline.gdsl</code></a> file in the <code class="language-plaintext highlighter-rouge">src/</code> folder and importing my project as a new Groovy project into IntelliJ (<em>File > New > Project from Existing Sources…</em>), a message popped up: <em>DSL descriptor file has been change and isn’t currently executed</em>.</p>
<p><img src="/images/2016-08-12-jenkins-pipeline-autocompletion-in-intellij/DSL-descriptor-file.png" alt="message that DSL descriptor file is found" /></p>
<p>Since then, I can enjoy autocompletion as well as well as documentation of the Pipeline DSL.</p>
<p><img src="/images/2016-08-12-jenkins-pipeline-autocompletion-in-intellij/autocompletion.png" alt="autocompletion" />
<img src="/images/2016-08-12-jenkins-pipeline-autocompletion-in-intellij/documentation.png" alt="documentation" /></p>
<p>It is not completely clear to me, why I was not able to achieve that with my existing IntelliJ project. Adding the <code class="language-plaintext highlighter-rouge">src/pipeline.gdsl</code> file did not trigger any message. So probably, the Groovy setup that I had was not 100% correct. During import of the new project, IntelliJ found the two directories contained in my global library.</p>
<p><img src="/images/2016-08-12-jenkins-pipeline-autocompletion-in-intellij/import.png" alt="message that DSL descriptor file is found" /></p>
<p>I hope this helps, either by making you aware of this “feature” or by helping to solve the problem that the <code class="language-plaintext highlighter-rouge">.gdsl</code> file is not picked up.</p>
<p><strong>EDIT January, 13th:</strong> <a href="http://lihsmi.ch">Michael Lihs</a> figured out one more caveat on <a href="http://stackoverflow.com/questions/41062514/use-gdsl-file-in-a-java-project-in-intellij">StackOverflow</a> (quoting his answer):</p>
<blockquote>
The problem was, that /src was not marked as a source root folder in my project. Creating a folder /src/main/groovy, putting the file in there and marking it as a sources root (right click on the folder -> Mark directory as -> Sources Root) did the trick.
</blockquote>
<p><a href="//st-g.de/2016/08/jenkins-pipeline-autocompletion-in-intellij">Jenkins Pipeline Autocompletion in IntelliJ</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on August 12, 2016.</p>
//st-g.de/2016/08/virt-manager-in-vagrant
//st-g.de/2016/08/virt-manager-in-vagrant2016-08-11T08:00:00+02:002016-08-11T08:00:00+02:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p><a href="https://virt-manager.org/"><em>Virt-Manager</em></a> is a graphical user interface for <em>libvirt</em>, a popular toolset for managing virtual machines provided by KVM.
Unfortunately, it does not work well on MacOS.</p>
<p>The <a href="https://github.com/jeffreywildman/homebrew-virt-manager">MacOS port</a> did not work for me, as I was not able to connect to the socket on the remote host even after changing adding the <code class="language-plaintext highlighter-rouge">socket=/var/run/libvirt/libvirt-sock</code> parameter. The same holds for its command-line pendant <code class="language-plaintext highlighter-rouge">virsh</code>.</p>
<p>So the idea to let it natively run on Linux inside a Vagrant VM came up.</p>
<h2 id="running-virt-manager-in-vagrant">Running Virt-Manager in Vagrant</h2>
<p>On that path, I had to overcome the following two challenges:</p>
<ul>
<li>to run a window manager inside the VirtualBox GUI</li>
<li>copying over my SSH private key into the VM</li>
</ul>
<p>#
After digging into Vagrant’s configuration options, I found solutions for both of these issues.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="n">config</span><span class="p">.</span><span class="nf">ssh</span><span class="p">.</span><span class="nf">forward_agent</span> <span class="o">=</span> <span class="kp">true</span>
<span class="n">config</span><span class="p">.</span><span class="nf">ssh</span><span class="p">.</span><span class="nf">forward_x11</span> <span class="o">=</span> <span class="kp">true</span></code></pre></figure>
<ul>
<li>SSH agent forwarding allows to use the private key located on my Mac</li>
<li>X11 forwarding allows to run graphical applications inside the VM and display them using <a href="https://www.xquartz.org/">XQuartz</a> as a normal program window on my Mac.</li>
</ul>
<h2 id="ready-to-go-vagrantfile">Ready-to-Go Vagrantfile</h2>
<p>I’ve put the complete <code class="language-plaintext highlighter-rouge">Vagrantfile</code> on Github: <a href="https://github.com/StephenKing/vagrant-virt-manager">StephenKing/vagrant-virt-manager</a>.</p>
<p>After cloning the repository, the VM can be created using</p>
<figure class="highlight"><pre><code class="language-shell" data-lang="shell"><span class="nv">$ </span>vagrant up</code></pre></figure>
<p>and <code class="language-plaintext highlighter-rouge">virt-manager</code> can be started using</p>
<figure class="highlight"><pre><code class="language-shell" data-lang="shell"><span class="nv">$ </span>vagrant ssh
ubuntu@ubuntu-xenial:~<span class="nv">$ </span>virt-manager</code></pre></figure>
<h2 id="ssh-host-key-verification">SSH Host Key Verification</h2>
<p>The only issue that remains for me is the acknowledgement of the KVM host’s SSH fingerprint. The solution instead was to once connect to the server from the command line:</p>
<figure class="highlight"><pre><code class="language-shell" data-lang="shell"><span class="nv">$ </span>vagrant ssh <span class="nt">--</span> <span class="s2">"ssh-keyscan server.example.com >> ~/.ssh/known_hosts"</span></code></pre></figure>
<p><a href="//st-g.de/2016/08/virt-manager-in-vagrant">virt-manager on MacOS</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on August 11, 2016.</p>
//st-g.de/2016/07/heat-assign-floating-ip-address-to-each-instance-of-an-autoscaling-group
//st-g.de/2016/07/heat-assign-floating-ip-address-to-each-instance-of-an-autoscaling-group2016-07-27T08:00:00+02:002016-07-27T08:00:00+02:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>OpenStack <a href="http://docs.openstack.org/developer/heat/">Heat</a> is the orchestration project of OpenStack, which allows to describe and provision OpenStack resources based on Heat <em>templates</em>, similar to <a href="https://aws.amazon.com/cloudformation/">Amazon CloudFormation</a>.
For a research project, I recently had the requirement to assign a <em>floating</em> IP address to every instance of a Heat <em>autoscaling group</em> (ASG).
Given the fact that such setup is contradictory to the common practice having a load balancer in front of an ASG, this costed me some time to figure out.</p>
<h2 id="creating-a-single-instance-using-heat">Creating a Single Instance Using Heat</h2>
<p>Setting up a single instance using a Heat template (HOT) is pretty easy, straightforward and also covered in the <a href="https://github.com/openstack/heat-templates/blob/497cf075f70e3b7051fcf52301cd8e356260a37c/hot/servers_in_existing_neutron_net.yaml">Heat templates repository</a>:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="c1"># single_server_with_floating_ip.yaml</span>
<span class="c1"># (abbreviated example based on hot/servers_in_existing_neutron_net.yaml)</span>
<span class="na">parameters</span><span class="pi">:</span>
<span class="c1"># ~~~snip~~~</span>
<span class="na">network</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">string</span>
<span class="na">description</span><span class="pi">:</span> <span class="s">The network for the VM</span>
<span class="na">constraints</span><span class="pi">:</span>
<span class="pi">-</span> <span class="pi">{</span><span class="nv">custom_constraint</span><span class="pi">:</span> <span class="nv">neutron.network</span><span class="pi">}</span>
<span class="na">public_net</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">string</span>
<span class="na">description</span><span class="pi">:</span> <span class="s">ID of public network for which floating IP addresses will be allocated</span>
<span class="na">constraints</span><span class="pi">:</span>
<span class="pi">-</span> <span class="pi">{</span><span class="nv">custom_constraint</span><span class="pi">:</span> <span class="nv">neutron.network</span><span class="pi">}</span>
<span class="na">resources</span><span class="pi">:</span>
<span class="na">server1</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">OS::Nova::Server</span>
<span class="na">properties</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">Server1</span>
<span class="na">image</span><span class="pi">:</span> <span class="pi">{</span> <span class="nv">get_param</span><span class="pi">:</span> <span class="nv">image</span> <span class="pi">}</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">port</span><span class="pi">:</span> <span class="pi">{</span> <span class="nv">get_resource</span><span class="pi">:</span> <span class="nv">server1_port</span> <span class="pi">}</span>
<span class="na">server1_port</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">OS::Neutron::Port</span>
<span class="na">properties</span><span class="pi">:</span>
<span class="na">network_id</span><span class="pi">:</span> <span class="pi">{</span> <span class="nv">get_param</span><span class="pi">:</span> <span class="nv">network</span> <span class="pi">}</span>
<span class="na">security_groups</span><span class="pi">:</span> <span class="pi">[{</span> <span class="nv">get_resource</span><span class="pi">:</span> <span class="nv">server_security_group</span> <span class="pi">}]</span>
<span class="na">server1_floating_ip</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">OS::Neutron::FloatingIP</span>
<span class="na">properties</span><span class="pi">:</span>
<span class="na">floating_network_id</span><span class="pi">:</span> <span class="pi">{</span> <span class="nv">get_param</span><span class="pi">:</span> <span class="nv">public_net</span> <span class="pi">}</span>
<span class="na">port_id</span><span class="pi">:</span> <span class="pi">{</span> <span class="nv">get_resource</span><span class="pi">:</span> <span class="nv">server1_port</span> <span class="pi">}</span></code></pre></figure>
<p>As we can see, the <code class="language-plaintext highlighter-rouge">server1_floating_ip</code> (of type <a href="http://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Neutron::FloatingIP"><code class="language-plaintext highlighter-rouge">OS::Neutron::FloatingIP</code></a>) is NAT’ed to the <code class="language-plaintext highlighter-rouge">server1_port</code> which in turn is assigned to the server instance (type <a href="http://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Nova::Server"><code class="language-plaintext highlighter-rouge">OS::Nova::Server</code></a>) that we instantiate.</p>
<h2 id="creating-an-autoscaling-group">Creating an Autoscaling Group</h2>
<p>In order to create an ASG of multiple instances, each only with a private IP, we also find help in the <a href="https://github.com/openstack/heat-templates/blob/master/hot/asg_of_servers.yaml">Heat templates repository</a>:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="c1"># abbreviated example based on hot/asg_of_servers.yaml</span>
<span class="na">parameters</span><span class="pi">:</span>
<span class="c1"># ~~~snip~~~</span>
<span class="na">image</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">string</span>
<span class="na">description</span><span class="pi">:</span> <span class="s">Name or ID of the image to use for the instances.</span>
<span class="na">network</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">string</span>
<span class="na">description</span><span class="pi">:</span> <span class="s">The network for the VM</span>
<span class="na">default</span><span class="pi">:</span> <span class="s">private</span>
<span class="na">resources</span><span class="pi">:</span>
<span class="na">asg</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">OS::Heat::AutoScalingGroup</span>
<span class="na">properties</span><span class="pi">:</span>
<span class="na">resource</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">OS::Nova::Server</span>
<span class="na">properties</span><span class="pi">:</span>
<span class="na">key_name</span><span class="pi">:</span> <span class="pi">{</span> <span class="nv">get_param</span><span class="pi">:</span> <span class="nv">key_name</span> <span class="pi">}</span>
<span class="na">image</span><span class="pi">:</span> <span class="pi">{</span> <span class="nv">get_param</span><span class="pi">:</span> <span class="nv">image</span> <span class="pi">}</span>
<span class="na">flavor</span><span class="pi">:</span> <span class="pi">{</span> <span class="nv">get_param</span><span class="pi">:</span> <span class="nv">flavor</span> <span class="pi">}</span>
<span class="na">networks</span><span class="pi">:</span> <span class="pi">[{</span><span class="nv">network</span><span class="pi">:</span> <span class="pi">{</span><span class="nv">get_param</span><span class="pi">:</span> <span class="nv">network</span><span class="pi">}</span> <span class="pi">}]</span>
<span class="na">min_size</span><span class="pi">:</span> <span class="m">1</span>
<span class="na">desired_capacity</span><span class="pi">:</span> <span class="m">3</span>
<span class="na">max_size</span><span class="pi">:</span> <span class="s">10</span></code></pre></figure>
<p>We see the <a href="http://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Heat::AutoScalingGroup"><code class="language-plaintext highlighter-rouge">AutoScalingGroup</code></a> defined with the Nova server instance being scaled up and down (between 1 and 10 instances).
Using the <a href="https://github.com/openstack/heat-templates/blob/497cf075f70e3b7051fcf52301cd8e356260a37c/hot/asg_of_servers.yaml#L61-L75">scale up and down</a> URLs provided, one can test the functionality of auto scaling.</p>
<p>Recalling our goal, namely to assign a floating IP address to every instance of the ASG (and also to the ones created during scale-out), it looks intuitively right to extend the <code class="language-plaintext highlighter-rouge">resource</code> block of the previous template. However, only one <code class="language-plaintext highlighter-rouge">resource</code> can be given to the <a href="http://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Heat::AutoScalingGroup"><code class="language-plaintext highlighter-rouge">AutoScalingGroup</code></a>. For our use case, we also need the floating IP resources to be added and destroyed on demand.</p>
<h2 id="scaling-a-complete-stack">Scaling a Complete Stack</h2>
<p>The solution to my problem was finally hidden in one of the other examples, the <a href="https://github.com/openstack/heat-templates/blob/497cf075f70e3b7051fcf52301cd8e356260a37c/hot/asg_of_stacks.yaml#L47-L51">ASG of stacks</a>.
Essentially, Heat can also scale a complete stack (defined by its own template file).
So what we now do is essentially to treat the server with its floating IP as defined in the first listing (<code class="language-plaintext highlighter-rouge">single_server_with_floating_ip.yaml</code>) as entity of scale and pass through all parameters:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="c1"># asg_of_server_with_floating.yaml</span>
<span class="c1"># (abbreviated example based on hot/asg_of_stacks.yaml)</span>
<span class="na">parameters</span><span class="pi">:</span>
<span class="c1"># ~~~snip~~~</span>
<span class="na">image</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">string</span>
<span class="na">description</span><span class="pi">:</span> <span class="s">Name or ID of the image to use for the instances.</span>
<span class="na">network</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">string</span>
<span class="na">description</span><span class="pi">:</span> <span class="s">The network for the VM</span>
<span class="na">constraints</span><span class="pi">:</span>
<span class="pi">-</span> <span class="pi">{</span><span class="nv">custom_constraint</span><span class="pi">:</span> <span class="nv">neutron.network</span><span class="pi">}</span>
<span class="na">public_net</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">string</span>
<span class="na">description</span><span class="pi">:</span> <span class="s">ID of public network for which floating IP addresses will be allocated</span>
<span class="na">constraints</span><span class="pi">:</span>
<span class="pi">-</span> <span class="pi">{</span><span class="nv">custom_constraint</span><span class="pi">:</span> <span class="nv">neutron.network</span><span class="pi">}</span>
<span class="na">resources</span><span class="pi">:</span>
<span class="na">asg</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">OS::Heat::AutoScalingGroup</span>
<span class="na">properties</span><span class="pi">:</span>
<span class="na">resource</span><span class="pi">:</span>
<span class="c1"># this refers to the file previously described</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">server_with_floating.yaml</span>
<span class="na">properties</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="pi">{</span> <span class="nv">get_param</span><span class="pi">:</span> <span class="nv">image</span> <span class="pi">}</span>
<span class="na">network</span><span class="pi">:</span> <span class="pi">{</span> <span class="nv">get_param</span><span class="pi">:</span> <span class="nv">network</span> <span class="pi">}</span>
<span class="na">public_net</span><span class="pi">:</span> <span class="pi">{</span> <span class="nv">get_param</span><span class="pi">:</span> <span class="nv">public_net</span><span class="pi">}</span>
<span class="na">min_size</span><span class="pi">:</span> <span class="m">1</span>
<span class="na">desired_capacity</span><span class="pi">:</span> <span class="m">3</span>
<span class="na">max_size</span><span class="pi">:</span> <span class="s">10</span></code></pre></figure>
<p>As the <code class="language-plaintext highlighter-rouge">type</code> of of resource, we can also supply a file name. That’s.. let me think.. <em>not</em> obvious?
After long search, was able to find this behavior documented under <a href="http://docs.openstack.org/developer/heat/template_guide/composition.html#use-the-template-filename-as-type">Template composition</a>.</p>
<p>Mind the <code class="language-plaintext highlighter-rouge">public_net</code> parameter defining the public network ID, from which floating IPs are assigned.
Once we trigger a scale-out, we can see antoehr instance including a floating IP being added after some seconds:</p>
<p><img src="/images/2016-07-25-heat-assign-floating-ip-address-to-each-instance-of-an-autoscaling-group/screenshot.png" alt="screenshot horizon" /></p>
<p>I am not aware of a way to launch such stack using the Horizon dashboard, except when the inner template is referenced via an HTTP(S) URL.
Instead, one can launch it using the command line client:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>openstack stack create my-test-stack <span class="nt">--template</span> asg_of_server_with_floating.yaml <span class="nt">-e</span> environment.yaml</code></pre></figure>
<p>All definition of the required parameters, i.e., image and network IDs, are happening in the <a href="https://gist.github.com/StephenKing/13987089075af2cb72d43fee4a0c1ef4#file-environment-yaml"><code class="language-plaintext highlighter-rouge">environment.yaml</code></a> file.</p>
<p>While my use case was a little academic (why in the world should something scale inside an OpenStack cloud and be <em>directly</em> reachable from the outside world), I think this (to me still unknown) concept of <a href="http://docs.openstack.org/developer/heat/template_guide/composition.html#use-the-template-filename-as-type">template composition</a> helps solving many more problems.</p>
<p>The complete code can be found in <a href="https://gist.github.com/StephenKing/13987089075af2cb72d43fee4a0c1ef4">this gist</a>.<br />
<small>(cover image by <a href="https://pixabay.com/en/clouds-nature-clouds-form-295695/">sipa on pixabay.com</a>)</small></p>
<p><a href="//st-g.de/2016/07/heat-assign-floating-ip-address-to-each-instance-of-an-autoscaling-group">Heat ASGs with Floating IP per Instance</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on July 27, 2016.</p>
//st-g.de/2012/11/typo3-at-universities
//st-g.de/2012/11/typo3-at-universities2012-11-21T23:00:00+01:002012-11-21T23:00:00+01:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>During the last two days, I had the opportunity to participate in a meeting of a special type of TYPO3 users: German universities.</p>
<p>Thanks to <a href="http://www.rrzn.uni-hannover.de/">RRZN / University of Hannover</a>, around 50 people from allover Germany met for an event called <a href="http://typo3-tagung.uni-hannover.de/">“TYPO3 an Hochschulen”</a> to talk about various topics regaring the use of TYPO3 in universities.</p>
<p>Besides a couple of known faces, a lot of people, who where new to me, participated and presented various kinds of topics in two parallel tracks. The type of talk varied, besides introductions to Extbase/Fluid or Git, showcases about Solr/Nutch or Shibboleth integration, also some interesting case studies of the whole TYPO3 ecosystem at the presenter’s university made their way into the schedule. It is interesting to see TYPO3 installations having around 50,000 active pages - or people administrating 150 similar, but different installations in a small team. You know, each institute in a university expects its special treatment ;-)</p>
<p>Together with Helmut Hummel, we had a talk about TYPO3 CMS 6.0 and how universities can participate and influence the development of TYPO3 (in our opinion) (<a href="http://www.slideshare.net/StephenKing/typo3-cms-60-und-einblicke-in-die-typo3entwicklung">slides</a>). The main message of my part of the talk was: inspiring people to share!</p>
<p>I think the meeting is a very good step into the direction of more cooperation and collaboration between the universities and with the whole non-academic community. More frequent regional meetings have been discussed in the closing session, as they already exist in <a href="http://jweiland.net/aktuell/artikel/2-treffen-der-typo3-webmaster-an-hochschulen.html">Baden-Württemberg</a>.</p>
<p>Prior to the meeting, the organizers had to limit the number of participants to one participant per university (and I guess a few were left out completely). As this was the first Germany-wide event of this kind, the huge interest showed the general interest in cooperation within this sub-community. I think future events will be planned having the large number of interests in mind.</p>
<p>Furthermore, I hope that I will meet a lot of people again during next year’s triage of <a href="http://typo3.org/community/events/">TYPO3camps or TYPO3 Developer Days</a>. That was one of my messages, as I think, like it happens to me during every event, all those people also travel home with a lot of new inspirantion, solutions, and new friends.</p>
<p>The feedback that we got was also really nice, often interesting, but also often expected: Having an LTS version is crucial for most universities and 6.0 is a big fear. To my knowledge, there’s often little own development of extensions going on inside the universities theirselves. So what I heard was really not surprising. TYPO3 CMS 6.0 brings a lot of changes, but also stays compatible in a lot of ways - that’s one of the benefits of TYPO3 CMS that we don’t throw all API away with each version :-)</p>
<p>The need for long running versions of TYPO3 does not only exist in unversities. I can understand that it is hard to do updates once or twice per year, when there are like 2000 editors that need to be instructed in the changes. So the majority of universities clearly wants to stay on LTS versions (if I got it right).</p>
<p>I hope the lengthy, final discussion about how to stay in touch will lead to some action (discussion were about a closed or open mailing list, or better a forum, and what to (not) discuss there, etc.). I think the biggest consensus was to have an official, open list on typo3.org infrastructure (like typo3.ug.german-universities I would suggest). So let’s see, whether we (=<a href="http://typo3.org/teams/server-team/">Server Admin Team</a>) get the request within the next days in order to create one (our address is admin(at)typo3.org btw. <em>hint</em> <em>hint</em>).</p>
<p>So finally, thanks to Thomas Kröckertskothen and especially Martina Ahlswede for their initiative to host this event (and sorry for myself playing “Tante aus Marokko” (*) and the weekly cycle of registering and then cancelling my participation). I’m looking forward to see such events also in the future!</p>
<p>Event web site: <a href="http://typo3-tagung.uni-hannover.de">http://typo3-tagung.uni-hannover.de</a></p>
<p>(*) <em>the German version of “She’ll Be Coming ‘Round the Mountain”</em></p>
<p><a href="//st-g.de/2012/11/typo3-at-universities">TYPO3 at Universities</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on November 21, 2012.</p>
//st-g.de/2011/12/a-retrospect-of-half-a-year-as-server-team-member
//st-g.de/2011/12/a-retrospect-of-half-a-year-as-server-team-member2011-12-27T23:00:00+01:002011-12-27T23:00:00+01:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>I am a member of the official TYPO3 Server Administration team since the Developer Days this year. With this article I not only want to look back, but also share some insights in what we are / I am doing in this team.</p>
<p>I have not planned this article at all, but there’s the funny thing called Twitter and after <a href="http://twitter.com/#!/T3RevNeverEnd/status/151749320067133440">Olivier’s tweet</a> last night, I decided to shed some light on our work. The question was:</p>
<quote>where can i find info about the actual team and its goals, roadmap and actions?</quote>
<p>So here is a short story of how I came into that team, why it is still a lot of fun, and what my plans for the next few months are.</p>
<h2 id="it-all-started-with-git">It all started with Git..</h2>
<p>It happened <a href="http://news.typo3.org/news/article/typo3-code-sprint-berlin-2011-t3csb11/">back then in Berlin</a>, when we (=Core Team) just made the switch to Git and Gerrit as version control system. Sitting in the crappy AO Hostel’s Teacher Room together with <a href="http://www.niekom.de/">Peter Niederlag</a> and defining all the review processes by writing the <a href="http://wiki.typo3.org/Git">developer documentation</a> somehow made me feel responsible for this topic. Improving procedures and sharing my knowledge by helping others while setting up Git - thus enabling them to contribute - took a lot of time.</p>
<p>Few weeks afterwards, we tried to set up Infrastructure Meetings. The aim was to improve the development infrastructure and processes of TYPO3 Core development. We all were no admins, thus had no possibility to change many things except documentation.</p>
<h2 id="the-typo3-developer-days">The TYPO3 Developer Days</h2>
<p>During the <a href="http://t3dd11.typo3.org/">TYPO3 Developer Days 2011</a> in Sursee this year, Susanne Moog and me have been asked to join the Server Admin Team. We both accepted and participated in the first physical Server Team meeting that was ever held (which is in fact no problem, as we all know us personally quite well). It was also when the picture above was taken. Usually we’re not so bad-tempered - it was just the “and now look serious” picture that got selected..</p>
<h2 id="time-went-by-and-work-started">Time went by.. and work started</h2>
<p>The major things I achieved since then:</p>
<ul>
<li>TYPO3 (Association) uses an <a href="http://www.otrs.org/">OTRS</a> Trouble Ticket system to distribute and document mail correspondence, like all the mails sent to/from {info,admin,documentation,..}@typo3.org etc. I was familiar with OTRS before and I really understood, why they all were very unhappy with it. OTRS is a lot like TYPO3. If you have a lazy admin, you will hate it. “lazy” in this case especially means “not familiar with disabling 80% of unneeded functionality or how to set better defaults”.</li>
<li>TYPO3 infrastructure uses OpenVZ for server <a href="http://wiki.openvz.org/Main_Page">virtualization</a>. OpenVZ is a very lightweight virtualization and you might now its commercial pendant Virtuzzo. I knew several other techniques, but this one was new for me. So getting familiar with it was one of my first actions. I must say that it is really the right tool for us.</li>
<li><em>“If it isn’t monitored, it is not in production.”</em> - some of you might now this speech. Although setting up a monitoring system was already planned (but not yet realized), I took over that job. It was just so essential for me to have it now. We are using <a href="http://www.zabbix.org/">Zabbix</a> since then to monitor the whole TYPO3 server infrastructure.</li>
<li><a href="https://buzz.typo3.org/teams/server-admin/article/inspiring-people-to-communicate/">Going live</a> with the BigBlueButton server.</li>
<li><a href="https://buzz.typo3.org/teams/server-admin/article/inspiring-people-to-collaborate/">Coordinating</a> the setup of the iEtherpad server.</li>
<li>In general, I tried to improve the visibility of the Server Team, by setting up this blog, creating a <a href="http://twitter.com/TYPO3server">twitter account</a>. In fact, this is why I’m writing this article: Make people know that we exist and that we are active - and that we try to help them.</li>
<li>Having a look, whether <a href="http://www.opscode.com/chef/">Chef</a> (or Puppet) fits us. Chef is for infrastructure automation and configuration management. It’s a big project and we’re just so far that we have a basic setup deployed to a handful of less-critical servers. But I try to set up new services based on Chef recipes (Zabbix and iEtherpad have been deployed with Chef already). Such a tool can enable you to easily configure your infrastructure - or to take it down with one wrong command. Thus it will still take some time during which we/I get used to it, before having 100% coverage (which we will probably never have).</li>
</ul>
<p>These have been my big projects, for which I account more than a full day of work.</p>
<h2 id="thousands-of-small-things">Thousands of small things</h2>
<p>You get never bored as an admin (maybe with Chef you do after a while?). Just a few examples, why the whole work is very diverse and involves a lot of communication with other teams and people:</p>
<ul>
<li>Helping teams while getting a new virtual server and setting it up (e.g. for Documentation Team’s manual rendering, Release Team’s Introduction Package template server, ..)</li>
<li>Helping Christopher with taking over the job as Wiki admin, updating it and <a href="https://buzz.typo3.org/teams/server-admin/article/typo3-wiki-now-with-single-sign-on/">integrating Single Sign-On</a>.</li>
<li>Exchanging an SSL certificate of the SVN server and see <a href="http://www.redmine.org/">Redmine</a> (<a href="http://forge.typo3.org/">forge</a>) crashing. Of course, it’s an annoying job - but afterwards it’s “Steffen vs. Redmine - 1:0” (or just a 1:4? ;-)).</li>
<li>Blocking DoS attacks kind of every two weeks from the forge server. We’re about to improve that..</li>
<li>Setting it up and communicating with guys at <a href="http://www.tue.nl/">TU Eindhoven</a>, which sponsored us a new server.</li>
</ul>
<p>Not all services are running on our infrastructure or maintained by us. There’s e.g. DNS (operated by <a href="http://www.snowflake.ch/">Snowflake</a>) or Mail plus a lot of other, mostly older things, which are operated by <a href="http://punkt.de/">punkt.de</a> admins. I think we have a pretty good balance between “do it on our own” and “using it from someone (trusted)”. It feels good, when you can just ask them and they fix it.</p>
<h2 id="plans-for-the-next-6-months">Plans for the next 6+ months</h2>
<p>Some parts of this are my personal agenda, some are (hopefully) plans of the whole team. As we have a very wide range of responsibilities (from system administration / virtualization up to PHP or even Ruby software administration) every one of us has a bit different working packages. This should just show that there would be a place maybe for you in this team, too. As in every TYPO3 team, there is enough work and enough space for new personalities bringing more spirit and ideas (and man/women-power).</p>
<p>So here are my major goals for the next months. If you are interested in helping or have knowledge in your head or your company, I / we would be happy to get supported!</p>
<ul>
<li>Bring back biweekly team meetings.</li>
<li>Finish project “F”. I just try to support and push Tim forward to finish that shiny thing that hopefully a lot will love. Something related to Mailing lists.</li>
<li>Upgrade OTRS to upcoming version 3.1. We sticked to the old-fashioned 2.x series.</li>
<li>Bring Chef forward to have a basic setup on every server. Deploy new services through Chef.</li>
<li>Deploy more monitoring scripts for Zabbix through Chef. Until now we’re limited to builtin checks.</li>
<li>Set up a centralized rsyslog server.</li>
</ul>
<p>From short this article got long.. so if you read this, it looks like you are really interested! Do you want to join our team? Then please get in touch with us (admin(at)typo3.org). You know that you can also just come up with service suggestions for the community that you maintain - running on infrastructure maintained by us?</p>
<p>Be warned: It’s all unpaid work, except maybe getting a reward out of the <a href="http://association.typo3.org/financial-statements/">Server Team’s budget</a>. If that’s fine for you and you like team-working and new challenges, feel welcome!</p>
<p><a href="//st-g.de/2011/12/a-retrospect-of-half-a-year-as-server-team-member">A retrospect of half a year as Server Team member</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on December 27, 2011.</p>
//st-g.de/2011/12/profiling-typo3-with-xhprof
//st-g.de/2011/12/profiling-typo3-with-xhprof2011-12-20T23:00:00+01:002011-12-20T23:00:00+01:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>There occur situations when your TYPO3 is awful slow and you have to figure out, why this is the case. Using a PHP profiler is the heavy hammer to gain the insights you need in this case.</p>
<p>Xdebug brings a PHP debugger and profiler. While I personally love the debugger together with PHP storm, the profiler output also needs a dedicated tool to browse through its results, e.g. KCachegrind or, again, PhpStorm.
So for profiling I learned to love XHProf, a PHP profiler developed by Facebook. Its advantage is that it has a PHP interface to show the results, which is nice to use and easily to integrate into TYPO3, as you will see in this article.</p>
<h2 id="installing-the-php-extension">Installing the PHP Extension</h2>
<p>Like Xdebug, XHProf also requires loading an own PHP extension. If your hoster doesn’t allow it, do it locally. Otherwise you’re able to profile even live sites, e.g. one of a million hits, based on a GET parameter, or whatever you want. That’s the nice thing that you can control this through PHP.</p>
<p>There is a PECL package xhprof, which you have to install, in order to build the PHP extension. On your development Mac you can also install php5-xhprof, in case you are using MacPorts. If installation through PECL, please refer to this blog post, from which I took my setup.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">pecl config-set preferred_state beta
pecl <span class="nb">install </span>xhprof</code></pre></figure>
<p>To load the extension, add these lines to your php.ini</p>
<figure class="highlight"><pre><code class="language-ini" data-lang="ini"><span class="nn">[xhprof]</span>
<span class="py">extension</span><span class="p">=</span><span class="s">xhprof.so</span>
<span class="py">xhprof.output_dir</span><span class="p">=</span><span class="s">"/var/tmp/xhprof"</span></code></pre></figure>
<p>The data collected during a profiler run will be saved to this output_dir.</p>
<p>Restart your web server and check with phpinfo() that XHProf is listed!</p>
<h2 id="xhprof-results-interface">XHProf Results Interface</h2>
<p>Now download the XHProf PHP files from the <a href="http://pecl.php.net/get/xhprof-0.9.2.tgz">PECL site</a> and extract them to wherever you want in the site’s document root or accessible through a separate virtual host.</p>
<p>We will assume that they reside in <code class="language-plaintext highlighter-rouge">/var/www/xhprof.example.com/</code> and are reachable under <code class="language-plaintext highlighter-rouge">http://xhprof.example.com</code>. Opening <a href="http://xhprof.example.com/xhprof_html/">http://xhprof.example.com/xhprof_html/</a> should then give you a “No XHProf runs specified in the URL.” message.</p>
<p>This vhost needs read access to the previously configured <code class="language-plaintext highlighter-rouge">output_dir</code>, I guess. So be careful with <code class="language-plaintext highlighter-rouge">open_basedir</code> restrictions.</p>
<h2 id="activating-the-profiler">Activating the Profiler</h2>
<p>In order to really run the profiler, you have to include this code, which starts debugging:</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><span class="k">include_once</span> <span class="s1">'/var/www/xhprof.example.com/xhprof_lib/utils/xhprof_lib.php'</span><span class="p">;</span>
<span class="k">include_once</span> <span class="s1">'/var/www/xhprof.example.com/xhprof_lib/utils/xhprof_runs.php'</span><span class="p">;</span>
<span class="nb">xhprof_enable</span><span class="p">(</span><span class="no">XHPROF_FLAGS_CPU</span> <span class="o">+</span> <span class="no">XHPROF_FLAGS_MEMORY</span><span class="p">);</span></code></pre></figure>
<p>As this is not sufficient, you also have to include some code to really safe the collected data:</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><span class="nv">$profiler_namespace</span> <span class="o">=</span> <span class="s1">'typo3'</span><span class="p">;</span> <span class="c1">// namespace for your application</span>
<span class="nv">$xhprof_data</span> <span class="o">=</span> <span class="nb">xhprof_disable</span><span class="p">();</span>
<span class="nv">$xhprof_runs</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">XHProfRuns_Default</span><span class="p">();</span>
<span class="nv">$run_id</span> <span class="o">=</span> <span class="nv">$xhprof_runs</span><span class="o">-></span><span class="nf">save_run</span><span class="p">(</span><span class="nv">$xhprof_data</span><span class="p">,</span> <span class="nv">$profiler_namespace</span><span class="p">);</span></code></pre></figure>
<p>To make inclusion easier, we create two files for later inclusion:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">/usr/share/php5/utilities/xhprof/header.php</code></li>
</ul>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><span class="cp"><?php</span>
<span class="k">if</span> <span class="p">(</span><span class="nb">extension_loaded</span><span class="p">(</span><span class="s1">'xhprof'</span><span class="p">))</span> <span class="p">{</span>
<span class="k">include_once</span> <span class="s1">'/var/www/xhprof.example.com/xhprof_lib/utils/xhprof_lib.php'</span><span class="p">;</span>
<span class="k">include_once</span> <span class="s1">'/var/www/xhprof.example.com/xhprof_lib/utils/xhprof_runs.php'</span><span class="p">;</span>
<span class="nb">xhprof_enable</span><span class="p">(</span><span class="no">XHPROF_FLAGS_CPU</span> <span class="o">+</span> <span class="no">XHPROF_FLAGS_MEMORY</span><span class="p">);</span>
<span class="p">}</span>
<span class="cp">?></span></code></pre></figure>
<ul>
<li><code class="language-plaintext highlighter-rouge">/usr/share/php5/utilities/xhprof/footer.php</code></li>
</ul>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><span class="cp"><?php</span>
<span class="k">if</span> <span class="p">(</span><span class="nb">extension_loaded</span><span class="p">(</span><span class="s1">'xhprof'</span><span class="p">))</span> <span class="p">{</span>
<span class="nv">$profiler_namespace</span> <span class="o">=</span> <span class="s1">'typo3'</span><span class="p">;</span> <span class="c1">// namespace for your application</span>
<span class="nv">$output_url</span> <span class="o">=</span> <span class="s2">"http://xhprof.example.com/"</span> <span class="c1">// keep the trailing slash</span>
<span class="nv">$xhprof_data</span> <span class="o">=</span> <span class="nb">xhprof_disable</span><span class="p">();</span>
<span class="nv">$xhprof_runs</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">XHProfRuns_Default</span><span class="p">();</span>
<span class="nv">$run_id</span> <span class="o">=</span> <span class="nv">$xhprof_runs</span><span class="o">-></span><span class="nf">save_run</span><span class="p">(</span><span class="nv">$xhprof_data</span><span class="p">,</span> <span class="nv">$profiler_namespace</span><span class="p">);</span>
<span class="c1">// url to the XHProf UI libraries (change the host name and path)</span>
<span class="nv">$profiler_url</span> <span class="o">=</span> <span class="nb">sprintf</span><span class="p">(</span><span class="nv">$output_url</span> <span class="mf">.</span> <span class="s1">'xhprof_html/index.php?run=%s&source=%s'</span><span class="p">,</span> <span class="nv">$run_id</span><span class="p">,</span> <span class="nv">$profiler_namespace</span><span class="p">);</span>
<span class="nv">$styles</span> <span class="o">=</span> <span class="s1">' style="display: block; position: absolute; left: 5px; bottom: 5px; background: red; padding: 8px; z-index: 10000; color: #fff;"'</span><span class="p">;</span>
<span class="k">echo</span> <span class="s1">'<a href="'</span><span class="mf">.</span> <span class="nv">$profiler_url</span> <span class="mf">.</span><span class="s1">'" target="_blank" '</span> <span class="mf">.</span> <span class="nv">$styles</span> <span class="mf">.</span> <span class="s1">'>Profiler output</a>'</span><span class="p">;</span>
<span class="p">}</span>
<span class="cp">?></span></code></pre></figure>
<p>Now go and modify the <code class="language-plaintext highlighter-rouge">index.php</code> of your TYPO3 installation to look like this:</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><span class="cp"><?php</span>
<span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">share</span><span class="o">/</span><span class="n">php5</span><span class="o">/</span><span class="n">utilities</span><span class="o">/</span><span class="n">xhprof</span><span class="o">/</span><span class="n">header</span><span class="mf">.</span><span class="n">php</span>
<span class="k">require</span><span class="p">(</span><span class="s1">'typo3_src/index.php'</span><span class="p">);</span>
<span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">share</span><span class="o">/</span><span class="n">php5</span><span class="o">/</span><span class="n">utilities</span><span class="o">/</span><span class="n">xhprof</span><span class="o">/</span><span class="n">footer</span><span class="mf">.</span><span class="n">php</span>
<span class="cp">?></span></code></pre></figure>
<p>You will hopefully end up with a shiny red box on the bottom left. Use the link there to browse through the collected results. If something fails, double-check the entered paths and URLs.</p>
<p><img src="/images/2011-12-profiling-typo3-with-xhprof/screenshot.png" alt="Screen shot" /></p>
<h2 id="further-tuning">Further Tuning</h2>
<ul>
<li>Instead of modifying the <code class="language-plaintext highlighter-rouge">index.php</code>, you could also set PHP’s <code class="language-plaintext highlighter-rouge">auto_(prepend|append)_file</code> options to <code class="language-plaintext highlighter-rouge">header.php</code> and <code class="language-plaintext highlighter-rouge">footer.php</code>. This might be useful for other applications than TYPO3 or if you e.g. want to profile the backend with its one million entry points (<code class="language-plaintext highlighter-rouge">alt_doc.php</code> etc).</li>
<li>To selectively run the profiler, modify the header and footer file and add your own conditions.</li>
<li>If you have installed GraphViz on your server, you can get nice call graphs, like <a href="https://buzz.typo3.org/fileadmin/user_upload/callgraph.png">this one</a>.</li>
</ul>
<p>More detailed information - also how to read the output - can be found in the original blog post by Lorenzo Alberton, where I got the idea and setup from: <a href="http://techportal.ibuildings.com/2009/12/01/profiling-with-xhprof/">techportal.ibuildings.com</a>. He earns most credits for my post. Please read that article, there’s a lot more to learn!</p>
<p>If further problems occur, please ask through the <a href="http://lists.typo3.org/cgi-bin/mailman/listinfo/typo3-dev">typo3.dev</a> mailing list.</p>
<p><strong>EDIT Jan 2013</strong>: In the meantime, two additional GUIs are known to me:</p>
<ul>
<li><a href="https://github.com/sandstorm/Plumber">Plumber</a> by Sebastian Kurfürst is a TYPO3 Flow package (<a href="http://www.youtube.com/watch?v=vIhaeLAyPW8">YouTube video</a> from T3CON12).</li>
<li>I’ve been asked by Gajus Kuizinas to refer to his alternative GUI: <a href="http://xhprof.io/">xhprof.io</a>. I haven’t tested it, yet, but it also looks very nice and useful.</li>
</ul>
<p>I’m happy to receive feedback what you like more!</p>
<p><a href="//st-g.de/2011/12/profiling-typo3-with-xhprof">Profiling TYPO3 with XHProf</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on December 20, 2011.</p>
//st-g.de/2011/04/doing-filename-checks-securely-in-PHP
//st-g.de/2011/04/doing-filename-checks-securely-in-PHP2011-04-03T00:00:00+02:002011-04-03T00:00:00+02:00Steffen Gebert//st-g.dest+blog-spam@st-g.de<p>Recently a security issue in TYPO3 <a href="http://typo3.org/teams/security/security-bulletins/typo3-sa-2010-022/">has been fixed</a>, where it was possible to circumvent checks, which should ensure file names to match specific patterns (e.g. denying .php file extensions to be uploaded or renamed to).</p>
<p>As this problem is heavily caused by PHP’s laxity, this blog entry aims to provide some explanations to you as developer to prevent you from placing similar security holes in your software or TYPO3 extensions.</p>
<p>Imagine you are programming a very basic file upload script and use the following piece of code:</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><span class="nv">$filename</span> <span class="o">=</span> <span class="nv">$_POST</span><span class="p">[</span><span class="s1">'filename'</span><span class="p">];</span>
<span class="nv">$contents</span> <span class="o">=</span> <span class="nv">$_POST</span><span class="p">[</span><span class="s1">'contents'</span><span class="p">];</span>
<span class="nb">file_put_contents</span><span class="p">(</span><span class="nv">$filename</span><span class="p">,</span> <span class="nv">$contents</span><span class="p">);</span></code></pre></figure>
<p>It is obvious that this code enables attackers to place arbitrary files on the server. So passing <code class="language-plaintext highlighter-rouge">evil.php</code> as a file name and sending PHP code, an attacker can execute any PHP code on your server.</p>
<p>So you might want to prevent saving .php files and other executable extensions by checking the file extension:</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><span class="k">if</span> <span class="p">(</span><span class="mi">0</span> <span class="o">===</span> <span class="nb">preg_match</span><span class="p">(</span><span class="s2">"/(.*)\.(php|php3|php4|php5)$/i"</span><span class="p">,</span> <span class="nv">$filename</span><span class="p">)</span> <span class="p">{</span>
<span class="nb">file_put_contents</span><span class="p">(</span><span class="nv">$filename</span><span class="p">,</span> <span class="nv">$contents</span><span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">die</span><span class="p">(</span><span class="s1">'Invalid file name given'</span><span class="p">);</span>
<span class="p">}</span></code></pre></figure>
<p>By using a whitelist approach, you can limit your upload script to allow only image file extensions:</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><span class="k">if</span> <span class="p">(</span><span class="mi">1</span> <span class="o">===</span> <span class="nb">preg_match</span><span class="p">(</span><span class="s2">"/(.*)\.(jpg|gif|png)$/i"</span><span class="p">,</span> <span class="nv">$filename</span><span class="p">)</span> <span class="p">{</span>
<span class="nb">file_put_contents</span><span class="p">(</span><span class="nv">$filename</span><span class="p">,</span> <span class="nv">$contents</span><span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">die</span><span class="p">(</span><span class="s1">'Invalid file name given'</span><span class="p">);</span>
<span class="p">}</span> </code></pre></figure>
<p>What I also read very frequently in the web is that file inclusions using the following code can be treated as “safe”:</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><span class="k">include</span><span class="p">(</span><span class="s1">'./themes/'</span> <span class="mf">.</span> <span class="nv">$_GET</span><span class="p">[</span><span class="s1">'filename'</span><span class="p">]</span> <span class="mf">.</span> <span class="s1">'.php'</span><span class="p">);</span> </code></pre></figure>
<p>But this code is far from being secure. The code snippet does not only expose a <a href="http://en.wikipedia.org/wiki/Directory_traversal">directory traversal</a> vulnerability (<code class="language-plaintext highlighter-rouge">../../../evil/more-evil</code>). Even the trailing <code class="language-plaintext highlighter-rouge">.php</code> will not secure the include. As arbitrary <code class="language-plaintext highlighter-rouge">.php</code> files placed on the server can be executed, you are already in trouble. Not only if you have <code class="language-plaintext highlighter-rouge">.php</code> files containing malicious code on your server, but also if <code class="language-plaintext highlighter-rouge">.php</code> files are included which are meant to be included in another context.</p>
<p>Still the list of possible ways to exploit this “safe” code is not finished. Also the code examples above which are supposed to do a proper validation of the user provided file name are vulnerable. Let me explain you why.</p>
<p>This is where control characters come in. The most well-known control characters are <code class="language-plaintext highlighter-rouge">\n</code> (carriage return) or <code class="language-plaintext highlighter-rouge">\t</code> (horizontal tab). Less frequently used (or known) in PHP is the <code class="language-plaintext highlighter-rouge">NUL(L)</code> character (or zero-byte char), which can be written as <code class="language-plaintext highlighter-rouge">0x00</code>, <code class="language-plaintext highlighter-rouge">chr(0)</code> or <code class="language-plaintext highlighter-rouge">"\0"</code> in PHP. In the first mentioned examples, <code class="language-plaintext highlighter-rouge">evil-file.php\0.jpg</code> could be submitted as value for <code class="language-plaintext highlighter-rouge">$_POST['filename']</code>.</p>
<p>Such a file name will pass the whitelist and the blacklist checks but <code class="language-plaintext highlighter-rouge">evil-file.php</code> will be used as target file name by <code class="language-plaintext highlighter-rouge">file_put_contents()</code>.</p>
<p>In the include example, an attacker could use <code class="language-plaintext highlighter-rouge">../avatars/user-1234.jpg\0rest-doesnt-matter</code> to execute the contents of the <code class="language-plaintext highlighter-rouge">user-1234.jpg</code> file. This file could have been previously uploaded as the attacker’s profile image in your imaginary web application or TYPO3 extension. Of course, the file could not contain image data, but malicious PHP code.</p>
<p>The remaining question is: Why do some functions, like the mentioned <code class="language-plaintext highlighter-rouge">preg_match()</code> or <code class="language-plaintext highlighter-rouge">substring()</code> work as expected and treat the <code class="language-plaintext highlighter-rouge">\0</code> as a normal character whilst other functions treat it as end of the string?</p>
<p>The reason is that PHP itself is programmed in C, which treats the <code class="language-plaintext highlighter-rouge">\0</code> as string terminator and ignores all following characters. And PHP relies on C’s (or the operating system’s) file system functions. That’s why only file system functions are affected by this problem.</p>
<p>A list of affected functions and some further reading can e.g. be found at <a href="http://www.madirish.net/?article=436">madirish.net</a>.</p>
<p>So keep in mind to also check for NUL character, when dealing with user-input, especially when passed to file system functions:</p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><span class="nv">$filename</span> <span class="o">=</span> <span class="nb">str_replace</span><span class="p">(</span><span class="nb">chr</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span> <span class="s1">''</span><span class="p">,</span> <span class="nv">$filename</span><span class="p">);</span></code></pre></figure>
<p>or using the <code class="language-plaintext highlighter-rouge">[[:cntrl:]]</code> class in <a href="http://php.net/manual/de/regexp.reference.character-classes.php">regular expressions</a></p>
<figure class="highlight"><pre><code class="language-php" data-lang="php"><span class="k">if</span> <span class="p">(</span><span class="nb">preg_match</span><span class="p">(</span><span class="s1">'/[[:cntrl:]]/'</span><span class="p">,</span> <span class="nv">$filename</span><span class="p">))</span> <span class="p">{</span>
<span class="k">return</span> <span class="kc">FALSE</span><span class="p">;</span>
<span class="p">}</span></code></pre></figure>
<p>The TYPO3 API provides two static functions in t3lib_div for file name and path checks:</p>
<ul>
<li><a href="http://api.typo3.org/typo3v4/current/html/classt3lib__div.html#a186b8fa7d4513e1fad65c06464d738a5"><code class="language-plaintext highlighter-rouge">validPathStr($filename)</code></a> checks a whole path to neither contain <code class="language-plaintext highlighter-rouge">'..'</code> (directory traversal) nor the <code class="language-plaintext highlighter-rouge">NUL</code> character</li>
<li><a href="http://api.typo3.org/typo3v4/current/html/classt3lib__div.html#a13ae2fa3e811f12987110f0d6b23e17b"><code class="language-plaintext highlighter-rouge">verifyFilenameAgainstDenyPattern()</code></a> verifies a single file name against the <code class="language-plaintext highlighter-rouge">fileDenyPattern</code>, which is <code class="language-plaintext highlighter-rouge">\.(php[3-6]?|phpsh|phtml)(\..*)?$|^\.htaccess$</code> by default.</li>
</ul>
<p>Usually this behavior to not react on control characters is called “binary-safe”. So you think that file systems function that are explicitly marked as binary-safe in PHP manual do not expose the mentioned problems? Unfortunately not. Especially <code class="language-plaintext highlighter-rouge">file_put_contents()</code> is explicitly denoted to be binary-safe. However, this only holds for the to-be-written file contents but not for the <code class="language-plaintext highlighter-rouge">$filename</code> parameter. This parameter has to be checked and secured by you as the programmer!</p>
<p><a href="//st-g.de/2011/04/doing-filename-checks-securely-in-PHP">Doing Filename Checks Securely (in PHP)</a> was originally published by Steffen Gebert at <a href="//st-g.de">Steffen Gebert</a> on April 03, 2011.</p>