Recent by Martijn de Jong 

Nginx as reverse proxy and SNI  

By Martijn de Jong | 11/10/23 4:43 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

I had some difficulty to find a good title for this article that would really cover the contents. Therefore, let me start with describing the problem I faced which led to this article. I have a lot of sites running on my home server (this blog being one of them) using different technologies. As I have a single IPv4 address, all these sites are behind a reverse proxy, for which I use Nginx. A couple of those sites are Domino sites and last week I realised there was something wrong in that area. I have several internet site documents on Domino for different urls. However, last week I realised that all my urls that were forwarded to Domino, were being serviced based on the same internet site document. In other words, Domino did not recognise for which internet site a request was meant.

Certificate Store: Submit vs Save  

By Martijn de Jong | 3/30/23 2:39 AM | Infrastructure - Notes / Domino | Added by Oliver Busse

I regularly receive question about the Certificate Store and CertMgr, which made me realise that there’s a lot of confusion around the Submit Request and the Save & Close buttons in the store and when to use what. Time for an article to hopefully solve some of that confusion.

On Domino thread IDs and Linux/Windows process IDs  

By Martijn de Jong | 3/1/23 9:53 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

A short tip on something which many people are probably not aware of, but which can be a huge time saver when you’re troubleshooting a Domino problem. As an example, see this error message from a Domino log: [062372:000014-00007F8001776700] 28/02/2023 13:16:20 CertStore: Error opening CertStore database [CN=PROD02/OU=SRV/O=ACME!!certstore.nsf] : The server is not responding. The server may be down or you may be experiencing network or VPN problems. Contact your system administrator if this problem persists. [062372:000014-00007F8001776700] 28/02/2023 13:16:20 CertStore: Error opening CertStore on [CN=PROD02/OU=SRV/O=ACME] : The server is not responding. The server may be down or you may be experiencing network or VPN problems. Contact your system administrator if this problem persists. Your first hunch might be that this is an error that’s caused by the CertMgr process. It’s related to the Certificate Store after all. But is this really the case?

Huddo Boards & Minio problems – Read before you restart!  

By Martijn de Jong | 11/21/22 2:06 AM | Infrastructure - Connections | Added by Roberto Boccadoro

Last week I got the unpleasant surprise of a no longer working Huddo Boards for Component pack installation at a customer after I had rebooted my Kubernetes environment. I had to reboot this environment after I updated the Kubernetes certificates. Of course, after a change you immediately think that your problem is related to the change you just made, but in this case the only relation was with the restart, which means that this can happen to everyone running Huddo/Kudos Boards for Component pack or Huddo Boards Docker.

Protecting your Domino container with fail2ban  

By Martijn de Jong | 11/7/22 4:25 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

If your Domino server is connected to the Internet, you’ll find that bots (hacked systems running a script) will throw a brute force attack on your Domino server. For me, especially, my SMTP server was under heavy attack. The reason why it’s interesting for hackers to find a valid login on an SMTP server, is that this will probably allow them to send spam through your mail server. Most mail servers allow sending mail through their servers for other domains for authenticated users only. The chances of them guessing any of the users in my Domino directory right and then also guessing the password correctly are basically zero, but the pollution of my log file is reason enough to stop them. Fail2ban is a very elegant program for Linux to do just that. You can configure it to scan log files for certain patterns (it uses RegEx to recognise them) and add hosts that match those patterns too often within a defined period of time, to the block list of iptables.

HCL Traveler and error 500  

By Martijn de Jong | 7/21/22 1:31 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

HCL Traveler is one of those addons for Domino that just works. If you have a properly configured HTTPS stack, you install it, start it and you’re basically done. From now on, you can connect your mobile devices to your Domino server to read your mail and calendar. At least, that has always been my experience until very recently. The other day I was sent to a customer to fix their problem with Traveler. They had upgraded their Domino server and Traveler installation from 8.5.3 FP5 to 12.0.1 FP1. Everything worked (Kudos for Domino!) except Traveler. Though on further discussion with the client it became clear that Traveler actually already broke earlier and hadn’t been working for the past 6 years or so.

Domino containers revisited   

By Martijn de Jong | 7/20/22 1:57 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

As I wrote in my last post about the Engage conference, a lot has happened in the Domino container space since I wrote my articles, as Daniel Nashed did some serious refactoring on all scripts, removing an insane amount of old code lines and adding some new functionality. This article will show the changes to the project compared to the time that I wrote the original 6-part series.

Working with standard Certificate Authorities in Domino 12  

By Martijn de Jong | 3/28/22 1:56 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

In the past weeks, I helped some colleagues with importing certificates in the Certificate Store of Domino 12 and while doing so, I noticed something peculiar. For many years, we haven’t had a proper way of creating certificates in Domino. The pre-12 database to create keys was completely outdated and didn’t allow for creating strong keys. As a result, most administrators got used to creating keys outside Domino, usually through on openssl command in Linux. This way of working found its way into procedures and many admins, instead of using the Certificate Store database, still follow these old procedures and create their keys outside Domino. I therefore decided to create a short article on how to create certificates with Domino 12 which are signed by a certificate authority which doesn’t support the ACME protocol.

Installing Tivoli/Security Directory Integrator on RHEL 8ì  

By Martijn de Jong | 1/14/22 11:21 AM | Infrastructure - Connections | Added by Roberto Boccadoro

On a new SDI 7.2 installation (with Java 8 and the latest Connections TDISOL directory for Java 8), I ran into a weird error: CTGDKG023E Error while starting main class.java.lang.reflect.InvocationTargetException .. Caused by: java.lang.UnsatisfiedLinkError: i4clntjni (Not found in java.library.path) Luckily, Google could help me on this one. This technote shows that if there are missing libraries, SDI doesn’t properly install and you will have to uninstall SDI, install the missing libraries, and reinstall SDI.

Expired certificate on your Kubernetes environment  

By Martijn de Jong | 12/7/21 2:36 AM | Infrastructure - Connections | Added by Roberto Boccadoro

Normally a Kubernetes environment is well maintained and regularly updated with the most recent versions of Kubernetes. However, with a Kubernetes environment that is just used a an HCL Connections Component pack installation, this might not be on your radar and it’s easy to let it just run attended. If you do that for too long though, like longer than one year, you’ll get into trouble

Domino-docker explained – Part 5 : Adding add-ons on top of your Domino image  

By Martijn de Jong | 11/2/21 2:34 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

In the previous parts, I explained how to create a Domino image and deploy it. But what if you want to add fix packs to your Domino image? Or Traveler, Volt or Verse? The scripts of the domino-docker project make this super simple. In this part, I’ll show you how to do this.

Domino-docker explained – Part 4 : The domino_container script  

By Martijn de Jong | 10/22/21 7:22 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

In the previous part, I showed how you can simply start and stop and open the Domino console with the domino_container command. This piece of script is responsible for interacting with the Domino container in a way where the average administrator doesn’t even have to realise that Domino is running inside a container. There are many more functions in this script that will help you manage your Domino server and in this part I will discuss them.

Domino-docker explained – Part 3 : Running your first Domino server in a container  

By Martijn de Jong | 9/30/21 10:56 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

In part 2 we created a Domino container image. Now we want to start the image. Of course, we could just use docker run <options> <imagename>, but with the scripts from the Domino Docker project, there’s a much easier option. In this part, I’ll show you what to do to make running, restarting and stopping images super easy.

Domino-docker explained – Part 2 : Creating your first Domino image  

By Martijn de Jong | 9/28/21 1:54 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

n the previous part, I looked at reasons why you might want to run your Domino server inside a container. In this part, I’m going to show how to create your first Domino image. We have to take one step back though, as since a couple of years, HCL provides their own docker image for Domino. So why would you want to create your own image? My experience is that it leads to a better image and it gives options to add your own tooling to the image. Nevertheless, using HCL’s image is an option and the script also provides an option to build on top of the standard HCl image. My advice: create your own.

Domino-docker explained – Part 1: Why run Domino inside a container?  

By Martijn de Jong | 9/28/21 1:51 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

In November 2018, Thomas Hampel (at that time still working for IBM) created the domino-docker github repository as an open source initiative to create scripts that would make it easier to run Domino inside a container. Even though the repository was started by IBM, the work was done by the community with most of the work done by one man in particular: Daniel Nashed. He contributed his Linux start/stop scripts to the project, but also wrote scripts to completely automate the build of the images. While working with the scripts, I realised two things: Daniel has built fantastic scripts to both build and run Domino containers With so much functionality added, the project didn’t manage to document this new functionality in detail With help from Daniel, I managed to build my own customised container and I experienced in the past months all the benefits from running Domino as a container, combined with the scripts from the Domino Docker project. However, if this project wants to get the attention it deserves, the documentation needs to be fixed and this is exactly what I’ll try to do in a series of 6 articles:

Domino 12 and Borg backup  

By Martijn de Jong | 4/20/21 4:39 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

I must admit that I’m quite excited about Domino 12. I was thinking this morning why actually. The new features in Domino 12 aren’t necessarily groundbreaking. They’re more about fixing things which should have been in the platform already, but were neglected by IBM in the years in which it would have been logical to implement them.

Domino 12 – SSL Performance  

By Martijn de Jong | 3/22/21 2:54 AM | Infrastructure - Notes / Domino | Added by Roberto Boccadoro

A few weeks ago I wrote about the new Certificate Manager in Domino 12, which enabled Domino 12 to request and automatically update LetsEncrypt certificates and implemented a better way of Server Name Indication (previously introduced in Domino 11.0.1), so you can use different SSL certificates for different websites without needing multiple IP addresses. The Certificate Manager also allows you to use the most recent (ECDSA) ciphers. The lack of this functionality in previous versions of Domino was an important reason why, in many Domino installations, an Nginx, Apache or IHS server is placed in front of the Domino HTTP task as a reverse proxy. There was however another reason: Domino used a lot of cpu power for and was rather slow to decrypt and encrypt SSL traffic. Letting Nginx/Apache/IHS offload the SSL de-/encryption task, reduced total load on the server and sped up performance. I therefore wondered if HCL also managed to solve this problem.

Domino V12 – The Certificate Manager  

By Martijn de Jong | 2/28/21 4:49 AM | Infrastructure - Notes / Domino | Added by Oliver Busse

HCL Domino V12 is in beta, and we currently have beta 2 to work with. One of the interesting new features of Domino V12 is the Certificate Manager task (certmgr). I’ve been playing around with this task and in this post I’ll tell about my experiences.

Installing the HCL Connections Component Pack 6.5 CR1 – Part 6: Configuring the applications  

By Martijn de Jong | 10/2/20 8:23 AM | Infrastructure - Connections | Added by Roberto Boccadoro

In part 5 I discussed the installation of all components. Now it’s time to configure them. My goal is not to duplicate the HCL documentation on this point, but to highlight where this documentation is ambiguous or incomplete. So by all means, also read that documentation.

Installing the HCL Connections Component Pack 6.5 CR1 – Part 4: Prepare the application environment  

By Martijn de Jong | 6/22/20 4:25 AM | Infrastructure - Connections | Added by John Oldenburger

If you followed the steps from the previous parts, you have a working Kubernetes cluster and a Docker registry containing all images necessary for installing the Component Pack. In this part I’ll discuss installing all helm charts up to, but not including, the actual components like OrientMe, Elasticsearch etc.

Installing the HCL Connections Component Pack 6.5 CR1 – Part 3: Uploading the images to a Docker registry  

By Martijn de Jong | 5/26/20 1:32 AM | Infrastructure - Connections | Added by Roberto Boccadoro

As a next step we’ll import the components of the Component pack into a Docker registry. If you have an existing registry in your company which you can use, you can skip right to the step where you upload the images in this repository. If you don’t, you’ll have the create the registry first.

Installing the HCL Connections Component Pack 6.5 CR1 – Part 2: Installing Kubernetes, Calico and Helm  

By Martijn de Jong | 5/19/20 2:34 AM | Infrastructure - Connections | Added by Roberto Boccadoro

In a series of articles I’m trying to fill the gaps in the HCL documentation regarding the Component Pack. In the first part I covered the installation and configuration of Docker. In this 2nd part I’ll cover the installation of Kubernetes together with Calico and Helm. After this the basic infrastructure is set up and the actual installation of the Component pack can begin. The installation I’m doing is a non-HA Kubernetes platform. One master and 2 worker nodes. If you need to setup a HA Kubernetes platform, you have to do a few extra steps, but using this manual and combining it with the HA documentation from HCL, you should be good.

Installing the HCL Connections Component Pack 6.5 CR1 – Part 1: Installing Docker   

By Martijn de Jong | 5/15/20 1:17 AM | Infrastructure - Connections | Added by Roberto Boccadoro

As I’m currently installing the HCL Connections 6.5 CR1 component pack at a customer I run into a lot of points where the HCL documentation is simply outdated or very confusing. In a series of articles I plan to write about the caveats in the documentation, to hopefully help you with your installation. In this first part I cover Docker.

Determining why records appear in your employee.error file in the TDI Assemblyline  

By Martijn de Jong | 3/16/20 2:14 AM | Infrastructure - Connections | Added by Roberto Boccadoro

Your TDI solution directory contains a couple of files which all start with employee. These files tell you which records were added, updated, deleted, skipped or produced an error. The last file is usually the most interesting one. Often employee records give an error because a user was deleted at an earlier stage and recreated with a different GUID. Sometimes users are created with the same login ID as a previous user, which was long deleted and there are more relatively obvious reasons like that why an employee record would create an error. However, sometimes you are just completely in the dark of why a record gives an error. That’s when this little tip helps you to determine the problem.

Decrypting a stash (.sth) file – Martijn's Blog  

By Martijn de Jong | 3/1/20 6:15 AM | Infrastructure - Notes / Domino | Added by Oliver Busse

HCL Domino saves it certificates in a .kyr file. IBM WebSphere saves it certificates in a Java Keystore / .jks format. Both formats allow you to save the password for the keystores in a stash file which has the extension .sth. The stash files allow you to do most actions without entering a password.

Useful SQL Queries for IBM Connections  

By Martijn de Jong | 6/30/18 7:03 AM | Infrastructure - Connections | Added by Oliver Busse

Over the past year, I’ve created my own set of useful SQL queries to get needed information and I thought I’d share them here. Please realise that my main client uses MS SQL for their backend, so the queries are written for that, though most should be universal or easy to change for DB2 or Oracle.