BarCamp Stuttgart 2019 (#bcs12)

BarCamp Stuttgart #12This weekend Saturday 14th and Sunday 15th of September the 12th yearly BarCamp Stuttgart took place again. As usual the event communications happens via twitter, see https://twitter.com/bcstuttgart hashtag #bcs12. I had paused for 5-6 years since last attending BCS. While the topic focus has shifted a bit since that time, it became more open and non-technical IMHO, it had been a really interesting and enjoying event again. I had the impression that this event is a place where some of the most motivated and engaged people from Stuttgart meet once a year. 

As usual, the topics on the open BarCamp had been very diverse but there were many new inputs and things to learn. The most technical session was probably on the Python scripting language, that answered some of the questions I had from my fight with python 2 versus 3 and virtualenv.
While the most physical and practical session has been Augen-Yoga:

A bit worrying is that the number of participants declined from last year’s 250. The Hospitalhof could have easily hosted more participants. So is the format of a BarCamp out of vogue eventually? It would be a pity as the organisation team did a great job again and in the contrary the BarCamp format could be used as a hack to the culture of a company potentially. No other format of event is so open, free and basic democratic that it could be an alternative or addition to corporate management updates or question and answer sessions.

So join next year again when BarCamp is again in Stuttgart or anywhere else!

OWASP SecurityRAT

SecurityRAT is a OWASP open-source project (github), the RAT stands for requirements automation tool. Currently the version 1 (1.7.8 as of the time of writing) is the productive version, but a version 2 is in the making with new architecture. SecurityRAT is based on JHipster Java rapid application development framework. Version 1 is a classic application, version 2 will be JHipster micro-service based.

SecurityRAT is used to create a set of security requirements for a supporting asset, the main part of a security concept. In the end it is nothing more than a replacement for MS Excel a way to get a filtered list of requirements with status and details provided by the development team on the implementation of the requirement in the asset. That status and the details are the important part, because having a list of requirements if fine but without knowing, whether they have been implemented or not and how, they don’t help a lot. Spending time on the details is important, they should contain additional information on the implementation, such as a link to the Jira or TFS issue, a link to a wiki page with implementation details or prove of implementation.

The cool thing is that one can define its own fields and values as long as it fits into the general idea of SecurityRAT. The central concept is a list of requirement skeletons classified by categories and tags that have columns and belong to a project type. The requirements in the database are skeletons or templates, from which instance of requirements for a given supporting assets will be created at run time. Those requirement instances are never actually stored in the database but exist only in memory on the client (browser) side. As soon as one has understood this, it’s a done deal in understanding how the tool works. You pour a pre-classified list of requirement templates into a database and instantiate them for a supporting asset at runtime with the additional benefit of filtering the list down to the relevant ones using category questions, filtering and tag selection at runtime.

We use SecurityRAT as an expert tool. This means, that not all developers work with the tool all day but only selected security lead experts, do. SecurityRAT spits out a Excel document with the requirement instances together with the up-to-date status and comment (optional columns). This is what other people work with. You put the Excel into the wiki for documentation or generate sub-tasks in Jira etc. SecurityRAT can also directly create stories and sync them bi-directionally, which would be really cool. Unfortunately this doesn’t work at my place, don’t ask why, it’s a sad big enterprise problem.

But working with Excel, or better CSV files, has some advantages, too. You can convert it to markup or generate task language in Jira from it easily with a little script. I use Groovy for it, but that’s a matter of personal taste.

SecurityRAT comes out of the box with a SQL dump for the OWASP ASVS (Application Security Verification Standard) requirements catalog. We have in the mean time at work also version 4.1 and many other catalogs that we pour into SecurityRAT instances. Version 1 somehow requires one RAT instance per catalog, although you can of course put multiple catalogs that have the same structure into one big instance, e.g. CIS Benchmarks. That ends up in a big list of instances for e.g.

  • OWASP ASVS
  • IEC 62443
  • DIN SPEC 27072
  • Corporate security requirement lists
  • CIS Benchmarks (you need to be member to get the XLS files)
  • Own catalogs e.g. for RabbitMQ

Apart from really filling in security requirements, SecurityRAT can be mis-used very well for other tasks. Things I use it for are among others:

  • Vulnerability assessment according to the OWASP Testing Guide (OTG). Excellent, you set the status to passed/failed and fill in the findings and get a nice Excel
  • Security maturity assessment according e.g. BSIMM or OWASP SAMM, answer the questions pre-filtered by level that should be achieved and get a nice Excel
  • Threat modelling using STRIDE – that stretches a bit the idea but works, when you have a list of threat skeletons instead of requirements.

Using SecurityRAT for status tracking with the OTG or ASVS are a good example where it makes sense to fill multiple, testing or requirement guides, into one instance, e.g. web services, mobile and IoT. The make a category or project types for these (one support asset can be either a web service or a mobile app or a IoT device). This way the user selects the type of asset in the initial “question” aka collection instance list implicitly.

In the end you could do a lot of this with Excel but you have IMHO the following advantages by SecurityRAT:

  • It’s not a document that rots somewhere on a share but a server with a nice web-based interface
  • The definable collection categories allow you to pre-filter the requirements at the beginning using customizable questions. Yes you could filter in Excel but you don’t have this very usable two-step process that helps in reality. Using Tags you can also filter when the requirements list has been generated, as well.
  • You can save the working results in a YAML file, load that again and continue, e.g. by adding also custom requirements. So only one place.

Being a server and database solution, filling an instance with data could either be done by UI, but you will quickly skip this idea for larger catalogs, even when the batch operations are really handy. Just use (again Groovy or other) scripts to convert a CSV source into SQL statements or directly insert it into the DB, which is then a bit more work. Unfortunately the entities in the SecurityRAT do not have surrogate keys so your script needs to manage the uniqueness of the database IDs by itself, which is sometimes, well a mess.

The down-side of the tool is a bit the missing calculation and missing colors for status fields that you have in a spreadsheet. E.g. for risk assessments it would be cool to calculate a risk factor from likelihood and impact automatically. But that is not possible.

BTW, SecurityRAT runs with docker out of the box, using MySQL or for license sake MariaDB is no problem. Problems will manifest themselves with endlessly long Spring Java exceptions, that will require a bit of digging into. We run everything in docker-compose, backup with mysqldump using docker exec and a nice landing page for the different instances.

Overall SecurityRAT, thumbs up!

The Practice of Network Security Monitoring

The second book from Richard Bejtlich in short time: “The Practice of Network Security Monitoring” has been read. This one is a bit newer, though not totally up to date, from 2014. The practical part of the book is based on the Security Onion (SO) distribution. Unfortunately a lot has happened with SO in the mean time. The book is still based on ELSA as part of SO, which has been swapped with the Elastic stack in the meantime. So the installation part could be skipped, also due to the fact, that I have already several times performed a SO installation at home. 

Just as with the TAO of network security monitoring book a lot of space is dedicated to various, in the mean time, well-known sniffing tools such as Wireshark, Bro, Argus, Sguil, Squert, Snorby etc. Nevertheless in the last third these tools are used for various real-life scenarios such as binary extraction with Bro, detecting server- and client-side intrusions, that were especially helpful. 

Security Onion is definitively the first choice for a real NSM with Sguil as real-time NSM console. For a home NSM a more historic Elastic stack-based NSM will probably be more useful, as I will not constantly monitor a NSM console all day long :-). The problem is a bit that SO is a big system, unfortunately a bit too heavy for the old laptop that I can dedicate to the NSM server part at the moment. Therefore I switched to SELKS, also a NSM distribution from Status Networks, also based on Elastic stack but a bit more light-weight. ELSA, based on syslog-ng doesn’t fit well anymore when you would like to use filebeat/packetbeat as logfile shipper. 

BSides Stuttgart 2019

This post is a bit delayed, on the weekend, 25th and 26 of May, the first BSides Stuttgart took place in the Wizemann location. I was lucky to have been there, because after monitoring the site months and weeks before, there was no program published and no way to buy tickets. But when looking on it 2 weeks before again, it was already sold out. As this was, as you can see, a Bosch-organized event, I still managed to get listed as a guest, thanks to dear colleague from Bosch CC. 

Security BSides conferences were originally a way to give those a platform whose presentations had been reject by the large conferences like DefCon or BlackHat, but in the mean time this is a grass-roots DIY conference format world-wide. And the contents are not second class in any way, in the contrary as this event has demonstrated!

BSides Stuttgart as the first of its kind in Stuttgart in 2019 took place in the previous industrial facility Wizemann co-working space. Same place as a digitalisation hackathon form Bosch before, just smaller. Great atmosphere and well prepared by CC security people from Bosch.

Co-organized by the ASRG (Automotive Security Research Group) and being hosted in Stuttgart, the event was pretty automotive oriented in general. BUT there was a general track with interesting presentations on cyber security in general. As you can image, this was the track I’ve been mostly following. 

Many colleagues from Bosch PSIRT and CERT and other (automotive) Bosch GBs attended the conference together with people from other companies such as Daimler.

These are the topics I had attended and are noteworthy on day 1:

  • How does ASCII and Unicode affect our Security
    Very interesting presentation on how Unicode and Punycode tricks can be used for DNS squatting and opening vulnerabilities for buffer overflows
  • Elastic Stack for Security Monitoring in a Nutshell
    Workshop on using ELK and Beats to build a SIEM more powerful than commercial products
  • OpSec++ the FastTrack
    Security testing using OSSTMM methodology
  • Cyber Threat Intelligence for Enterprise IT and Products
    A presentation from @Wagner Thomas Daniel (Bosch PSIRT)form PSIRT on a concept for product CTI
  • Weaponizing Layer 8
    How to treat users not as DAU but involve them into building a security culture in the organization.
  • Introduction to Osquery
    Very interesting workshop on osquery a service that exposes system information such as processes, filesystems, etc. via a SQLliste-compatible SQL interface. Also works with docker (as a companion to Sysdig?) and spits out logs.

On the second day, the sunny Sunday, I’ve been listening to the following presentations:

  • What to log? so many events, so little time
    On a tool from a Microsoft lady to catalog and filter the many events that the Windows OS produces with mapping to MITRE Att@ck techniques. Interesting approach and using sigma for generating SIEM queries for the relevant events. 
  • Security Onion
    Workshop on Security Onion, a Linux distribution specially for security monitoring, forensics and incident response, just like Kali is for pentesting.
    Included some real-live example how an attack could be detected and handled based on network logs using the various tools bundled in the distribution.
  • NoSQL Means no Security?
    Insights on the security posture and evolution of MongoDB, Redis and Elasticsearch. This will get us some ideas on hardening our NoSQL databases potentially.
  • Scale your Auditing Events
    Again from Elastic but on the Linux auditd sub-system and how to process its audit events with Auditbeat and Elastic stack for security monitoring.

Slides have been published on a bsidesstuttgart gitlab site or are posted on the bsidesstuttgart twitter

I’ve learned so many new tools, and new information especially in the areas of network security and security monitoring for getting OpSec started.

What’s pretty sure is that BSides Stuttgart will continue next year, maybe growing and giving also you a chance to grep a seat. I’s cool that we finally have a cheap and open security conference right here in Stuttgart, thanks to the organizers from Bosch for the great event! See you there next year, mark your calendar already for May 14 -16 2020!

The TAO of Network Security Monitoring

Wow, that was a thick book, the Tao of Network Security Monitoring, beyond intrusion detection from the guru of NSM, Richard Bejtlich. This book is considered the bible of NSM. The book is from 2004 and thus a bit out of date, especially as it is filled with tons and tons of tool, one will find that some of these do not yet exist anymore or development has stopped years ago. But the intention of the book is not to serve as a tool reference but to show which tools are available and what they can be used for. So the brain needs to translate the samples to what tools we have today available. And anyhow in each category we still have enough candidates.

The story line of the book is basically along the different types of network security monitoring data that one can capture along with the tools that provide it:

  • full content data (packet capture, e.g. from tcpdump, wireshark)
  • Packet headers
  • Session or flow data (e.g. from Argus, flow-tools)
  • Alert data (e.g. from Bro)
  • Statistic data

Bejtlich explains the use of these types of data and the corresponding tools using real-life samples of attacks. This is cool, although following the packet dumps without in-depth protocol knowledge of IP, UDP, TCP, DNS etc. is really a bit hard. Luckily he explains it after the printed dump, so one can be a bit lazy. Probably that is not a good idea, as one missing some learning, but that would probably require a second round of reading.

But the real learning from this book is understanding what a well-configured NSM system and especially stored session data can really give you to detect all kinds of attacks, if you just watch closely enough. The interesting question for me still to answer is how can I transfer this knowledge to cloud-based NSM, where we have some packet capture abilities but all the rest of the tools, how to make use of them in such an environment is left as an exercise. 

Summary: definitively worth a read, although it could get an update once a while.

IoT Hackers Handbook

Somewhere, I don’t know where, I was getting aware of the book “IoT Hackers Handbook” from Aditya Gupta. Well, bought it, read it. That wasn’t quite a long job as the font size is a bit larger than normal. There are two reasons you do this, either you want to avoid that older reader need their glasses (me?) or there’s not too much content but you still want to make it look like a in-depth book on the topic.

It was indeed a bit different than expected. Not bad, but different, which also tells you something. I’m a software guy, looking into hardware-near topics like BLE sniffing is interesting but not my homeland, so to say. But this book really started with hardware hacking after some introductory chapter on penetration testing IoT devices. I mean UART communication, JTAG debugging. Then it went slowly up in direction software, via firmware hacking, mobile apps (Android), software define radio (SDR) to Zigbee and BLE sniffing and packet resend. It didn’t get higher than this. That’s ok, as there had been topic, I hadn’t touched so far except for BLE sniffing. Especially the SDR part was quite interesting and encouraged my to maybe dig a bit into this topic. Understanding the communication of garage door openers etc. sounds interesting over all.

Don’t get me wrong, for consumer IoT devices, this is all important stuff to understand, test and hack. But IoT is a bit more than hardware, firmware and communication, at least in my mind. IoT lives from software, and not just hardware-near software. That is what brings the value and the new business models for IoT. Sure the book touched mobile apps as important part of a IoT solution but there is all the cloud connectivity and the software stack on the IoT device that I find the interesting part. And that was not covered beyond ZigBee and BLE. So not bad and helpful but surprising regarding the direction of what IoT pentesting should be like and maybe telling something about how IoT is regarded still today. 

To be fair, the book did dig into some use-cases of what you could do when having access to the device and being able to manipulate it at will, which wasn’t really difficult with the examples provided by the author. Weather stations, door openers, garage openers, the usual smart light bulb and beacons. I learned still a lot about tools and techniques for these low-end IoT devices and how easy it is to break them with just a little bit of knowing some tools and reading specifications. And unfortunately you can transfer this experience to more complex “IoT” devices like PLCs in IIoT or gateways. Just the specifications are a bit thicker and complex. But the door is equally wide open for white as well as black hats. 

HTTP Health Check for Docker

I just published a little tool called htcheck (docker-health-go project) on github: https://github.com/pklotz/docker-health-go . This being my first Go program, be kind to me, should there be better ways to implement it. 

It is intended as a very simple HTTP health check client for use in Docker health checks. Motivation, why this program?

When using docker directly or via docker-compose, you can and should define a health check, so that docker knows that the process it is running is doing well. There are a couple of libraries to provide a HTTP health endpoints for Go, such as [https://github.com/docker/go-healthcheck] and also Java offers with Spring Boot Actuator corresponding framework.

But on the client side, you still need to use curl or the outdated wget to perform the check. If you ever checked, which dependencies curl and thus libcurl4 brings with it, you might wonder if this is worth the ballast just to do a simple HTTP get with an exit code. Libcurl brings openldap libraries into the image and what not. So this little decent project provides a special-purpose HTTP client to use for health checks in docker or elsewhere instead of throwing a general-purpose HTTP client at the job.

It supports making a HTTP GET request to a URL and reading a JSON document back and checking for a value in it using a jq-like path expression.

Simple sample usage in Dockerfile:

COPY ./htcheck /usr/bin/

HEALTHCHECK --interval=5m --timeout=3s CMD htcheck -u http://localhost/ || exit 1

Sample usage for Spring Boot actuator health endpoint, which normally serve a JSON document in the form:

{
    "status" : "UP"
}
So using the JSON path feature, we can compare against a expected value:
COPY ./htcheck /usr/bin/

HEALTHCHECK --interval=5m --timeout=3s CMD htcheck -u http://localhost/health -p .status -v UP || exit 1

Licenses are checked and documented in the README. Thanks to the dependency projects [https://github.com/savaki/jq] and [https://github.com/spf13/pflag] that were made use of. Probably some features are still missing, but as a first shot, it should serve. S’il vous plaît!

Dualcomm 10/100/1000Base-T Gigabit Ethernet Network TAP

As I only have only a unmanaged switch Netgear FS116 at home, I don’t have a SPAN port to do network sniffing on the home LAN. In the course of building up a NSM (network security monitoring) setup for my home network, I needed a way to tap the wired LAN. Therefore I looked at network taps, which tend to be extremely expensive for home use. Finally I found some recommendation and bought a Dualcomm 10/100/1000Base-T Gigabit Ethernet Network TAP. It’s not cheap but better quality than a throwing star tap and offers full duplex passive sniffing of network traffic for affordable price. 

Setup is absolutely seamless, as there is no setup, just put the tap between the home router and the switch in order to get all internal traffic coming from outside, LAN cable to the sniffing ethernet interface and that’s it. The little box is powered by USB, so just put the USB cable to a monitor’s USB and that worked fine enough. 

Currently there are two options, either I use my RPI 3B LAN interface as sniffing interface and the RPI’s WLAN as management interface. Or I can also attach it to a old laptop, that I use as a monitoring collection station with SELKS distribution on it. I use SELKS instead of Security Onion (SO), because the laptop is just too old and SO freezes on this hardware. SELKS also has ELK stack and suricata installed and runs decently. Not optimal performance, but for testing it works. Also here WLAN is the management interface as the laptop also only has one wired LAN interface. And sniffing interfaces are not managed and don’t get an IP, as they are input only.

Long-term it could be interesting to replace the unmanaged switch with a managed switch so that one can move the tap to any other place  and use the SPAN port of the managed switch for e.g. the RPI. With the new RPI 4 model B one gets true gigabit LAN and that should be able to handle all traffic that the switch provides without any problems in such a home setup.

The packet-foo blog contains the probably best article series on network package capture and analysis including network taps, that you can find. 

Trying to build packetbeat for Raspberry PI (arm64)

After my previous article on building filebeat for Raspberry PI 3 B+ (arm64), I now wanted to get a binary for packetbeat, the second most interesting module of elastic beats. I tried the same approach with cross-compiling using GOARCH=arm64 but it fails, while a straight compile for amd64 works. It fails with a message that it excludes all Go files due to build constraints. Issue is that there is probably native C code involved and you cannot cross-compile this beat. I searched posts and tried all options for 2 hours, it does not work.

I tried again on the PI directly, the build is running but if you do a “go install”, it finally gets out of memory (“cannot allocate memory”). Problem is that the PI 3 has only 1 GB of memory and that does not seem to be enough. I tried all kinds of tricks, like setting GOMAXPROC=1, GOGC=70 but njet. The problem also seems to be related to the C build using gcc. You need to install “libpcap-dev” for the “pcap.h” header file using “apt-get install”, otherwise one gets a compile error earlier.  If using “go build -v -x” directly you get this “cannot allocate memory” message from gcc. When using “make” the build gets killed instead. Nevertheless it’s rare are there are reports from people that compile Kubernetes on arm64 RPI 3B like mine. Probably K8s does not contain native C parts like the libpcap in packetbeat. So I finally gave up, because …

But … there is good news ahead! Since a few days, the new Raspberry PI 4 Model B has been released! With up to 4 GB memory that will hopefully work. Also it has now true GB LAN, which is for network sniffing, not a bad idea either, when attaching it to a real network tap. So that is a clear buying plan for Juli!

Modsecurity Handbook

Modsecurity (from SpiderLabs) is probably the best known open-source web application firewall (WAF) originally (and still is) a module from the Apache web server. But in the mean time it is also available as module for Nginx (nginx-modsecurity) and IIS and other integrations. I came into contact with modsecurity in the context with Nginx.

The second important project in conjunction with modsecurity is the OWASP core rule set (CRS) a set of modsecurity rules for a WAF. You meet these two in many unexpected places, e.g. the Azure application gateway is based on Nginx with the CRS. Or the Kubernetes ingress controller is an Nginx with CRS and modsecurity WAF module included.

This is why I recently bought the book “Modsecurity Handbook” from Feisty Duck and the authors Christian Felini and Ivan Ristić (see https://www.feistyduck.com/books/modsecurity-handbook/).

This is really “THE” book on modsecurity from its authors, the bible to to say and goes into the depth of writing rules youself. It is not an explanation of the CRS, for this there no books, you have to read the rules in the github repository. This book does prepare you to do this, something that look daunting at first when you ever looked at the CRS ruleset without preparation.

That is probably one of the most important learnings one gets from the book, because I don’t know yet, whether I will write my own modsecurity rulesets myself. Although the second interesting insight were the use cases that you could cover with a WAF. The book has a long chapter on this topic and delves into detailed implementation ideas on use cases such as

  • IP address tracking and blacklisting
  • Session tracking, blocking, forced renegotiation and restricting session lifetime as well as detecting session hijacking (a well-known attack technique)
  • Brute force attack detection
  • Denial of service (DoS) detection
  • Periodic security testing and alerting
  • User tracking
  • Whitelisting of application operations
  • File inspection
  • Dynamic patching of application vulnerabilities or for exploits

The idea that nginx logs are an important source for security and audit logs for a SIEM is certainly not surprising. But being able to actively detect during runtime certain vulnerabilities and constantly reporting them as security alert is interesting. Think about missing security headers or wrongly configured content security policies (CSP) in HTTP. Instead of detecting it during vulnerability assessments or penetration tests, such inspection can happen during operations and thus provide a 100% coverage of all operations.

Also having a tool to quickly mitigate a vulnerability before the development team can come up with a fix and new release for a backend sounds interesting. You can even inject content into a response, e.g. Javascript. I just have some doubts about the complexity to introduce new rules and such mitigations quickly in environments where modsecurity and CRS are realistically found these days, such as Kubernetes ingress or WAF. An Azure application gateway for example does not expose the full functionality of modsecurity directly but hides most juwels under some own configuration portal.

At the end the book contains an extensive reference part with explanation of all the directives, variables, operators and actions of the modsecurity rule language. This way the book serves well when you need to actually develop rules in practice beyond what the Internet provides as reference resources.

Creating your own rulesets has its quite hard complexities in my opinion, but it is a tool in your defensive toolset. At least using the CRS with a WAF out-of-the box with just slight tuning, such as disabling rules that produce false positive or unneeded rulesets should be possible. That is anyhow the only thing that environments such as Azure application gateway allow you to do. Going beyond that needs a good reason. The disadvantage is really that such configuration is decoupled from the protected service or application and if we can fix a vulnerability there quickly it certainly is the preferable option before we turn to using custom modsecurity rules. In times of continuous deployment that should be fast enough to avoid dynamic patching. For old applications where there is no team anymore maintaining security or that has half-year release cycles this is still a valid option in the security control portfolio.

I’m curious if I will finally use it one time or not.

Yours, Peter

Ramen (ラーメン)

I hear the term “Ramen” already before and about the hype on Ramen shops all over the world but never payed attention to it. Now it happened a couple of weeks ago that, when dining out with my girl friend at Karl’s kitchen in Breuninger Stuttgart, they offered Ramen soup. Always open to sumptuous experiments, I tried … and was positively surprised. It tasted delicious.

Now what followed is, that’s my style, a thorough research about Ramen. Where does it come from and how to prepare it yourself. I quickly found out some basic receipts for the Japanese fast-food and set out to go practical. 

First of all, there are 3 steps to get a ramen soup:

  1. Base broth
  2. Spice broth
  3. Soup with toppings

When you watch receipts in youtube, you see that often for sake of efficiency the distinction between base and spice broth is ignored and one broth according to the local habits is prepared only. Here we want to stick to the original as close as we can.

Base Broth

First is to note, that ramen is used in all of Japan and this is a country that stretches from the sub-tropical south of Okinawa up to frozen north of Hokkaido. So it is natural that depending on region the basic style is different and adapted to what’s available there. This is why one distinguishes different ramen types:

  • Shōyu ramen (醤油, “soy sauce”) with soy sauce
  • Shio ramen (塩,”salt”) based on fish and seafood
  • Miso ramen (味噌) based on fermented soy beans (miso paste)
  • Karē ramen (カレー,”curry”) with curry
  • onkotsu ramen (豚骨ラーメン) based on pork meat and bones

You can find quite original receipts on Lecker (onkotsu ramen) and Chefkoch. For the first time, I did indeed follow the basis receipt with pork meat and bones, as that’s what I could get. Getting pork bones is rare actually as they do not keep up very well and you don’t use it for cooking sauce or broth normally but rather beef bones. So lucky coincidence.

So I used:

  • Pork bones with meat
  • Mixed vegetables for soup (mirepoix)
  • Garlic
  • Chicken wings
  • Kombu alga – hard to get because it is obviously expensive

For detailed preparation see the receipts, I had to exchange Kombu alga for Wakame alga. You cook for 2 hours. The broth can be frozen, taken care to not fill the bottles or other containers very full in order to avoid that they burst. 

Spice Broth

The spice broth is made, short term, from

  • Soy sauce
  • Flares of bonito (dried thin sliced tuna) – one can buy in the local Asia shop
  • The meat from the base broth

The spice broth does not have to be cooked long up-front but, when needed as it does not take too much time. The spice broth is added to the base broth to create the soup’s broth.

Soup and Toppings

Now you can start creating the soup itself. With ramen it’s like Pizza, you can add what you like if not following some traditional receipts. Here some ideas:

  • Sautéd Mushrooms are always good: Shiitake, Enoki or other asian mushrooms are must have. 
  • Spring onions, I like them sautéd as well
  • Pak Choi, again sautéd shortly
  • Sprouts, sautéd
  • Meat, fish from the broth or shrimp
  • Roasted vegetables like corn, thin sliced carrots
  • Cooked eggs
  • Pumpkin
  • Sesame paste

and of course noodles, either ramen noodles or other asian noodles like soba or udon. The ramen noodles are made of wheat, soba from buckwheat. Arrange everything neatly, with the half eggs on the top and ready is the ramen soup. It’s not really fast to prepare with all the stuff to roast and the hour-long broth cooking but well prepared you can re-use the frozen broth and then it’s not too much effort.

Delicious and good for a whole meal, enjoy!

And next time another variant …

Peter

Produkte digital-first denken

Barbara Hoisl, ist eine freiberufliche Business- und Strategieberaterin und eine lang-jährige Freundin aus alten Zeiten, als ich bei Hewlett-Packard (HP Openview Software, ein Bereich, den es in dieser Form nicht mehr gibt) gearbeitet hatte.  Barbara ist eine, nun ja visionäre, Expertin für Software-Produktmanagement, Finanzierung von Startups und den Software-Business. Ich hatte die positive Erfahrung Barbara früher bei HP eine kurze Zeit als Chefin zu haben. 

Letztes Jahr hat Barbara doch tatsächlich ein eigenes Buch geschrieben, “Produkte digital-first denken“, auf Deutsch. Ich schätze mich glücklich zu denjenigen zu gehören, die Anfang des Jahres eine (kostenlose) Ausgabe ihres neuen Buches bekommen hat. Daher wollte ich hier darüber berichten, wie das Buch geworden ist und was ich daraus gelernt habe.

Erst ist man irritiert, muss man ein deutsches Buch über das Thema Digitalisierung schreiben? Aber ich habe auch in der Arbeit schon öfter festgestellt, man vergisst schnell, dass ich Jahre-lang bei einer amerikanischen Firma gearbeitet habe und die Verwendung von English als Umgangssprache für mich zur Selbstverständlichkeit geworden ist, aber für doch noch viele, die nicht aus der Softwarebranche kommen eher noch ein Problem darstellt. Und ihr Buch wendet sich ganz klar an deutsche mittelständige Unternehmen, wo Deutsch doch noch die Fachsprache darstellt. Bis vor wenigen Jahren war das bei meinem Arbeitgeber (Bosch) auch noch der Fall.

Das ist auch schon einer der interessanten Punkte, warum dieses Buch eine Lücke im Portfolio der Bücher über Digitalisierung darstellt, es ist wirklich für den Personenkreis geschrieben, der die Digitalisierung und die Einführung von Softwareprodukten, IoT und IIoT durchführen muss um fit für die Zukunft zu werden. Und den Zielgruppen-gerechten Schreibstiel hat Barbara auf faszinierende Weise getroffen. Da sind auf der einen Seite doch die vielen anglophilen Ausdrücke, die für uns Softwerker so selbstverständlich sind, für das Zielpublikum aber hole Phrasen darstellen. Aber hinter den “Phrasen” stecken eben wesentliche Konzepte der Softwarewelt, welche die heute großen IT-Player (GAFA = Google Apple Facebook Amazon) eben erfolgreich gemacht haben und die ohne eine Anpassung der etablierten Produktionsfirmen in Deutschland in Zukunft auch deren Geschäft gefährden werden. Das heisst, wenn sie eben nicht die Digitalisierung und die Einführung von Softwareprodukten ernst nehmen.

Und genau das erklärt Barbara in verständlichen Worten, erklärt die Sätze wie “Software is eating the world”, “Winner takes it all” Effekt in Platform-Geschäftsmodellen, “Think big, smart small” und “Sell the future” Strategie. Interessant ist dabei, dass ich, der sich auch schon intensiv mit Software-Platform Geschäftsmodellen auseinandergesetzt habe und der all diese Prinzipien der Softwarewelt als gegeben und als klar versteht, dabei immer noch etwas lernen kann. Man wird sich über die Unterschiede zwischen den deutschen erfolgreichen Produktionsunternehmen und den (meist amerikanischen) IT-Unternehmen noch klarer und erkennt den Handlungsbedarf Produkte digital neu zu erschaffen.

Bosch ist eine solches Unternehmen, mit hunderten Produktionswerken und unglaublichem Wissen über Fertigung und Logistik und ein Unternehmen, dass sich ganz klar auf den Weg zum Software-Unternehmen befindet. Mein Geschäftsbereich “Bosch Connected Industry” ist an vorderster Front mit dabei. Aber ich habe auch schon, auf Messen oder in Gesprächen, bemerkt, dass dies durchaus nicht für die Masse der kleineren mittelständischen Unternehmen, insb. in Baden-Württemberg gilt. Dabei gibt es hier viele heutige Weltmarktführer in hunderten technischen Nischenmärkten. Und genau diese muss das Wissen über die wirkliche, Buzzword-erklärte Bedeutung erreichen. Barbara’s Buch ist einzigartig darin, genau das hoffentlich erreichen zu können.

Was mich dabei fasziniert hat ist, durch die Darstellung im Buch wieder klar zu werden, wie wichtig dabei die richtige Denkweise (“Digital Mindset”) zu bekommen (zu erlernen?). Zu verstehen wie die neuen großen innovativen IT-Player denken gegenüber den traditionellen etablierten aber langsamen Unternehmen. Barbara erklärt dabei viele Modelle, wie den Produkt-Lifecycle, Moore’s Law und exponentielles Wachstum, 3 Horizonte der Innovation, Innovator’s Dilemma, 10 Types of Innovation, 6D-Modell. Die beiden letzen waren z.B. auch für mich neu und ich habe mir gleich die dazugehörende Literatur besorgt.

Das schöne an ihrem Buch ist, dass sie die abstrakten Modell immer mit praktischen Beispiele aus B2B und B2C Märkten erklärt. Bosch Software Innovations (mein erster GB bei Bosch) kommt übrigens auch darin vor (sic!). Lieblingsbeispiel Tesla, wo es für mich auch noch etwas zu lernen gab. 

Schliesslich gibt sie auch noch einige Empfehlungen am Ende des Buchs wie man die Transition zu einem Unternehmen, das “digital-first” denkt organisieren sollte. Nicht, dass jedes Unternehmen das so angehen würde und man sieht die Probleme, die dadurch in der Umsetzung entstehen im eigenen Unternehmensbereich. Alles in allem eine bereicherndes Buch, dass ich jedem der im IIoT Bereich unterwegs sind oder sein sollten, und das sind eben alle traditionellen Produktionsunternehmen, wärmstens and Herz legen kann. 

Viele interessante Erkenntnisse beim Lesen!

Peter

Bulletproof SSL and TLS

As I’m currently involved with lots of openssl automation at work, I bought the book “Bulletproof SSL and TLS” from Ivan Ristić. See the book’s site at https://www.feistyduck.com/books/bulletproof-ssl-and-tls/. Attention, it looks like on Amazon there is only the 2014 edition available, while on Ivan’s blog (https://blog.ivanristic.com/2017/07/announcing-bulletproof-ssl-and-tls-2017-revision.html), which I found out after the purchase of course, there is a 2017 version mentioned. Nevertheless that the book edition I read was a bit dated, I learned a lot, despite having been engaged with openssl before. 

It is a difference being able to generate and sign some certificates and knowing the history, the vulnerabilities and mechanisms of the protocol itself. This book is definitively the “bible” of TLS from the founder of the (Qualys) SSL Labs with the famous SSL server test tool (btw. also available as standalone tool: ssllabs-scan on github). So there is quite some expertise and mastership behind this book.

What can one learn from the book? Well first a thorough basis and the insight, or maybe reminder, that TLS is not just encryption but also certificate-based authentication and provides integrity and session management. So it’s a bundle of security functionality that can be used not only for HTTPS but also any other protocol that you can run over TCP. There are many articles about TLS port forwarding, but with the book, I have finally gotten the differences. 

There is by the way also a github repository to the book that contains among other resources configuration files for setting up a own root CA for self-signed certificates. That being a task that I’m just involved in and this thus very handy to verify my configuration taken from other sources in the web. Clearly for public customer or browser-facing endpoints one will always have to use purchased certificates from a public CA. But in the innards of a system, behind a reverse proxy or from the application backend to a infrastructure service, such as RabbitMQ or a DB, self-signed certificates, well-configured, serve well their purpose. And you save money and have the full control over expiration time and what not.

Especially interesting for were the details on OSCP and OSCP stapling and all the other initiatives that there are. Certainly a topic that one would like to explore at work for getting an additional grain of security into especially cloud-hosted services. Another concept that was covered were the different ways of pinning and what it really means. It is not so a sophisticated concept that nobody uses, anyhow. 

What I found especially helpful were, beyond some openssl command-line examples also a in-depth chapter on configuring Nginx with TLS, something that I happen to just do at the moment at work, too. What a coincidence. That adds well to the Nginx TLS documentation, which is more reference than tutorial. Especially the securing of a down-stream connection to backend services in a reverse proxy scenario.

Well, this is thick book and it took a while to get through but it was worth it and I’m now feeling much better prepared for practical work with TLS, openssl and juggling with certificates.

Yours, Peter

Books for Learning

Books, printed books, despite the promise that e-books will kill them in the age of the Internet, they’re still there and sold in millions by Amazon and the like. But I observe, that they’re still on the decline. Not because everyone would only read them in electronic form only. For this the smartphone is simply not really convenient. You need a tablet/iPad to enjoy reading e-books IMHO. No, it’s because books aren’t used anymore for learning. 

When I look around me at work, nobody other than me has books besides his desk and seems to read any book for work. Maybe they only read prose for pleasure, fine as well. But even that you see less and less in public transportation. What I see is that people don’t use books for learning but use other media instead. They watch videos, listen to podcasts or read articles in the world-wide web as they need it. You have an issue, you search and find some resource that tells you how to get ahead. On-demand learning, so to say. 

A book is something longer-term, you buy, you watch is waiting for you and you spend weeks, if not months to digest it in one piece if it is a good one. That takes time and effort and persistence to do. That’s not like a two-page article or 20min video that comes right to the point. I guess this is really the point, these other internet-based media are easier to digest and solve your problem of learning that you have right in this moment.

But this type of learning is a shallow one. You don’t really learn the fundamentals of the technology or topic. You learn how to solve exactly this one problem and the next day you’re as dump as you were. This is why people always say “I have no idea”. Really I avoid this phrase like hell. I do want to have a clue, an idea on the topics, I speak about, otherwise I shut my mouth. 

And for this you need deep knowledge, expert knowledge. Books are written usually by experts, at least if it’s a good book. They build up the topic from the grounds and systematically consolidate the matter using examples and give you reasons and arguments. At the end you are maybe not an expert yourself, experience is missing. But at least you have the feeling that you have profound knowledge to start from. 

There is the model of “cone of learning” from Edgar Dale, I think. It explains how good media are for learning. The book is doing pretty bad in this model. It is passive learning and you remember only small parts of what you read. In contrast a video or podcast is remembered much more. And that is probably right in the general. How long do you remember what you read a year ago in a book? Nevertheless the depth is a different in a book in contrast to other media. and I would say it needs to stay in the learning mix also these days, electronic or not. 

There is one more aspect, that I reflected about. Exactly that is the point, writing a book is equally more effort and takes more time than creating a podcast, video or writing an article (like this one, lol). So there is so much preparation going into writing a (good) book. A good friend of mine, Barbara Hoisl, wrote a book last year. Content-wise this is a completely own article, but she worked for more than a year only on the book, left apart the time for thinking about it and preparing the steps to get to start at all. But this is not only time, it is reflection and thinking time. And a book includes this reflection and thinking of months and years. It is maybe courageous to talk about wisdom, but a book certainly captures more wisdom than other media. And this is why one should read books in addition to consuming internet-based media.

Well, think about it, yours

Peter

Installing filebeat on Raspberry PI 3 (amd64)

Currently I’m experimenting with using a Raspberry PI 3 B+ as a network security monitoring (NSM) sensor node. So I have Bro and Suricata installed on that little guy running Kali Linux for arm64. But I need a modern way to transport the logs to its log monitoring station. So not using syslog-ng or ryslog but the best log shipper for the elastic stack, and that is Beats, better the Filebeats.

Problem: Elastic does, unfortunately, despite desperate inquiries from users in the forums not provide binaries or a .deb package for Beats. After trying some other paths I came across some receipts to install Beats on arm64 by manually compiling the binary with Go. I have to say, Go is marvelous! On the PI itself, I had bad luck, because the “go build” quickly finished with out-of-memory. So that didn’t work unfortunately on that little pal.

But because Go is so cool, I just “cross-complied” it on a bigger laptop, also running Kali Linux. And that’s so easy that I have to tell the world, because the other receipts are sometimes too specific and parts are missing for a full running manual installation, which is more than just the filebeats binary.

Step one on the other Debian-base system, the laptop, you need of course also Go installed.

# mkdir -p go/src/github.com/elastic
# cd go/src/github.com/elastic
# git clone https://github.com/elastic/beats.git
# export GOPATH=$PWD/go

You could also get the sources with “go get” bu that doesn’t matter, result the same.
Now the important step, watch out:

# export GOARCH=arm64
# cd beats
# go build -v -x

Flags just so see what’s happening, as go build is very silent otherwise. Magic, in a few seconds, you have a “filebeat” binary in this directory!
Try:

# file filebeat
filebeat: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=svVi8LJGhqXEjRJveTrA/7cOYouMPn1VzyeJqwq3W/TXZ3DZ8Wa_QYdKnsR8cm/8bg35yoawYw18mAJ30oX, not stripped

Remember we are on amd64 not arm64 on the laptop!
Now just copy the file over to the PI using ssh and test it there:

# ./filebeat –help

Works! But when you try

# ./filebeat modules list

It does not show any, because we are missing something, all the module configuration and kibana dashboards that are normally also contained in the .deb package.  So on the laptop just install filebeats, as for amd64, there is of course package:

# cd ~/Downloads
# curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.1.1-amd64.deb
# dpkg -i filebeat-7.1.1-amd64.deb
# filebeat modules list

Here you get the modules of course.
Now just let’s see what’s in the debian package:

# dpkg –listfiles filebeat|more

As you can see, besides the binary (for amd64) no other binaries are really in the .deb, just lots of YAML and JSON files. Now that’s of course good news.
So what I did for getting a fully functional installation is just copy the files over form the laptop to the PI using SSH in /etc/init.d/filebeat, /etc/filebeat/*, /usr/share/filebeat, /lib/systemd/system and /usr/bin/filebeat (a script). Then place the compiled arm64 binary in “/usr/share/filebeat/bin/filebeat” and we’re got to go on the PI:

# filebeat modules list

And here we get the list.
Now this is not a package that will be manged by apt-get of course. Maybe, I didn’t try one could for to install the official amd64 .deb package and only exchange compiled binary.

Hope this helps, Peter

Book Platform Ecosystems

There are a couple of books on platform business models and ecosystems. I recently bought two of them. One is “Platform Ecosystems, Aligning Architecture, Governance and Strategy” from Amrit Tiwana (see cover). The book is built up very systematically in explaining what the difference between a platform business model in a multi-sided platform and a classical product or solution business in a single-sided model is. He stresses the important of the interaction between architecture and governance in the evolution of a platform. Something a platform software architect likes to hear of course. Some later chapters are too advanced for my current needs at the moment but very very informative and clearly written. This will be for me a very influential book and is definitively worth a read!

TRIZ

End of last year I read this book on the TRIZ methodology, in English “theory of creative problem solving” originally developed by Genrich Saulowitch Altschuller in the former soviet union. Quite some time already I wanted to read about TRIZ, as I’m fascinated by the idea that you could systematically find and direct innovation. The book is also a nice introduction to innovation management by the way. Of course the method comes more from mechanical engineering and unfortunately does not translate directly to my space of computer science. That remains the open question for me, is it possible to find and document similar patterns in software engineering as TRIZ does it? Or do we maybe have something like this with architecture and design patterns more or less? It was an interesting read nevertheless and gets you thinking about how to systematically find new ideas and solutions for problems. And of course it is a nice piece of off-the-path thinking from Russia!

New paknet blog

Hi,

in the past there were multiple blogs for private, software architecture and the unofficial MIDAS weblog. I decided to restart with one single blog that can potentially include material for all topics and probably new ones as well.

Don’t expect any more material on MIDAS though. First Blue Elephant Systems is dead, MIDAS as a product is dead and the whole topic of IT management is dead in my mind. You will potentially see references to MIDAS from Atos or 4Things Solutions. These companies try to market what is left from MIDAS to HP OM customer that did not notice yet that MIDAS is gone with its author. The last brain that knew anything about the complete product suite is me, and I’m not with Atos nor 4Things. And don’t believe anyone else that they understand the product nor further develop it nor be able to maintain its rests, just don’t. I will keep the old content of the blog in the archive, this is where it belongs.

I’ve changed work and also the area. No more IT management, monitoring or stuff that is just a necessary evil that needs to be done. IoT and industrial internet/i4.0 or connected industry as it’s called is now the new topic. And its thrilling and interesting like hell! This is the topic I will be writing about.

Peter

6. Jahrestagung DGSD

Since June 2012 blue elephant systems is member of the “Deutsche Gesellschaft für System Dynamics” (DGSD), German chapter of the “System Dynamics Society” (SDS), the institution for the advancement of the system dynamics methodology for modeling and simulation of complex dynamic systems. On the 21st to the 22nd of June 2012 the DGSD held its 6th yearly conference on system dynamics (SD) in Frankfurt Main in the rooms of the PA Consulting Group and I had been participating for the 2 days for the first time.

A series of presentations, mostly in the areas of business and jurisdiction showed applications of SD from purely qualitative to very data-centered quantitative models, from very high-level academic to pretty practical levels. For me the purpose was mainly learning about system dynamics, seeing practical SD models and their applications in various fields as well as getting contacts in the SD community. In that respect it was a very interesting and mind-opening event that got me started in modeling and simulation.
I am a the moment evaluating SD tools, from XJ Technologies AnyLogic, via Consideo MODELER to Vensim, so I also wanted to see what tools other people use. Apparently due to the excessive price of AnyLogic a lot of people use Vensim and Consideo, although those usability is not even remotely comparable to AnyLogic. Well we need to see how we can solve this problem.
As an interesting fact, I found out that the University of Stuttgart, with Prof. Dr. Meike Tilebein is very active in the field, whose doctor father is obviously the BWL Prof. Dr. E. Zahn. When I studied BWL as my minor field of study at university, Prof. Zahn had been my professor in Organization (in-depth elective).

I especially liked the openness of the SD people, it was obvious that they are happy of every new member that is deeply interested in system dynamics, and obviously blue elephant is bringing in a new application field where SD is not yet widely applied.