Sync to k8 native Secret not working via secretObjects

In the past several days, I’ve been dealing with ASCP (AWS Secrets and Config Provider) on EKS Kubernetes cluster. This AWS blog post has great detail on the usage.

Here’s some gotchas that I went through

  1. Enabling syncSecret

If you intend to expose secrets from AWS Secrets Manager item as Pod’s environment variables, it’s super important to set syncSecret.enabled to true when you install secrets-store-csi-driver. I did not turn it on when I tested it for the first time and struggled to sync SecretProviderClass to Kubernetes secrets resource. I followed many tutorials, but not many mentioned about it.

Later I found the solution when I read CSI driver documentation. Afterward I removed the existing helm chart and then reinstalled it with the feature enabled

2. Kubernetes secret resource not sync’ed

I expected a secret resource to be created when I create a SecretProviderClass resource. However, it was not created and I tried so many different things. Reinstalled secrets-store-csi-driver, recreated SecretProviderClass resources. None worked. I was thinking that the timing of Kubernetes native secret resource to be created was when the SecretProviderClass is created.

I was so wrong and later found this issue on the project issue page. It turned out that a Pod must mount the volume via SecretProviderClass, then a sync’ed secret resource gets created automatically.

Bulletproof WordPress via Nginx

I assume many developers consider WordPress as a joke since it’s made with “PHP”. However, WordPress is still powering a lot of websites. So quite often it is inevitable to do some work on a project that deals with WordPress.

Personally, I’ve had to deal with many WordPress sites and resolve security issues. The most common issues that I observed have been:

  • Backdoor attack to use the infected host to perform various types of attack
  • Stealing an admin cookie
  • Using the stolen cookie to post many dangerous posts
  • Using the stolen cookie to upload other scripts in wp-content directory
  • And so on

The most used attack paths the hackers/hacking tools seem to be wp-admin/(post-new|post|admin-post|).php and /wp-login.php.

Anyhow, the most impactful defense mechanism that I found was to whitelist IP address that belongs to a certain admin user. So far, nothing else has beaten that solution, so I call it a bulletproof Nginx config for WordPress site.

Here’s my Nginx config that I used for my clients to prevent hackers from attempting to intrude a WordPress site.

  location ~ /wp-admin/admin-ajax\.php$ {
    try_files $uri =404;
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_read_timeout 300;
    fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
    allow all;
  }

  location ~ (/wp-admin/.*\.php|wp-login\.php$) {
    try_files $uri =404;
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_read_timeout 300;
    fastcgi_pass unix:/var/run/php/php7.1-fpm.sock;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;

    ## whitelist IPs
    allow x.x.x.x;
    deny all;
    error_page 403 = @wp_ban;
  }

  location @wp_ban {
    rewrite ^(.*) https://mysite.com permanent;
  }

  location ~* /(?:uploads|files|wp-content|wp-includes|akismet)/.*.php$ {
    deny all;
    access_log off;
    log_not_found off;
  }

Downgrade to openjdk 8 for Jenkins on Ubuntu 18 Bionic

On my Ubuntu 18 Bionic server, I installed Jenkins. Upon completion of the installation, there was error message that goes something like this:

Output from sudo journalctl -xe

 Feb 19 14:10:17 macmini.shinstudio.lan jenkins[21270]: OpenJDK 64-Bit Server VM (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.4, mixed mode)
Feb 19 14:10:17 macmini.shinstudio.lan jenkins[21270]: Aborting
Feb 19 14:10:17 macmini.shinstudio.lan systemd[1]: jenkins.service: Control process exited, code=exited status=1
Feb 19 14:10:17 macmini.shinstudio.lan systemd[1]: jenkins.service: Failed with result 'exit-code'.
Feb 19 14:10:17 macmini.shinstudio.lan systemd[1]: Failed to start LSB: Start Jenkins at boot time.
-- Subject: Unit jenkins.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit jenkins.service has failed.
-- 
-- The result is RESULT.
Feb 19 14:10:18 macmini.shinstudio.lan systemd[1]: Reloading.
Feb 19 14:10:18 macmini.shinstudio.lan systemd[1]: Starting resolvconf-pull-resolved.service...
-- Subject: Unit resolvconf-pull-resolved.service has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit resolvconf-pull-resolved.service has begun starting up.
Feb 19 14:10:18 macmini.shinstudio.lan systemd[1]: Started resolvconf-pull-resolved.service.
-- Subject: Unit resolvconf-pull-resolved.service has finished start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit resolvconf-pull-resolved.service has finished starting up.
-- 
-- The start-up result is RESULT.
Feb 19 14:10:22 macmini.shinstudio.lan sudo[19303]: pam_unix(sudo:session): session closed for user root
Feb 19 14:10:28 macmini.shinstudio.lan kernel: [UFW BLOCK] IN=enp0s10 OUT= MAC=01:00:5e:00:00:01:f0:9f:c2:c6:9e:f4:08:00 SRC=192.168.1.101 DST=224.0.0.1 LEN=32 TOS=0x00 PREC=0xC0 TTL=1 I
Feb 19 14:11:19 macmini.shinstudio.lan sudo[21416]: shane : TTY=pts/0 ; PWD=/var/log/jenkins ; USER=root ; COMMAND=/bin/systemctl start jenkins
Feb 19 14:11:19 macmini.shinstudio.lan sudo[21416]: pam_unix(sudo:session): session opened for user root by shane(uid=0)
Feb 19 14:11:19 macmini.shinstudio.lan systemd[1]: Starting LSB: Start Jenkins at boot time...
-- Subject: Unit jenkins.service has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit jenkins.service has begun starting up.
Feb 19 14:11:19 macmini.shinstudio.lan jenkins[21419]: Found an incorrect Java version
Feb 19 14:11:19 macmini.shinstudio.lan jenkins[21419]: Java version found:
Feb 19 14:11:20 macmini.shinstudio.lan jenkins[21419]: openjdk version "10.0.2" 2018-07-17
Feb 19 14:11:20 macmini.shinstudio.lan jenkins[21419]: OpenJDK Runtime Environment (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.4)
Feb 19 14:11:20 macmini.shinstudio.lan jenkins[21419]: OpenJDK 64-Bit Server VM (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.4, mixed mode)
Feb 19 14:11:20 macmini.shinstudio.lan jenkins[21419]: Aborting
Feb 19 14:11:20 macmini.shinstudio.lan systemd[1]: jenkins.service: Control process exited, code=exited status=1
Feb 19 14:11:20 macmini.shinstudio.lan sudo[21416]: pam_unix(sudo:session): session closed for user root
Feb 19 14:11:20 macmini.shinstudio.lan systemd[1]: jenkins.service: Failed with result 'exit-code'.
Feb 19 14:11:20 macmini.shinstudio.lan systemd[1]: Failed to start LSB: Start Jenkins at boot time.
-- Subject: Unit jenkins.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit jenkins.service has failed.
-- 
-- The result is RESULT.
Feb 19 14:11:29 macmini.shinstudio.lan kernel: [UFW BLOCK] IN=enp0s10 OUT= MAC=01:00:5e:00:00:01:f0:9f:c2:c6:9e:f4:08:00 SRC=192.168.1.101 DST=224.0.0.1 LEN=32 TOS=0x00 PREC=0xC0 TTL=1 I
Feb 19 14:11:33 macmini.shinstudio.lan sudo[21467]: shane : TTY=pts/0 ; PWD=/var/log/jenkins ; USER=root ; COMMAND=/bin/journalctl -xe
Feb 19 14:11:33 macmini.shinstudio.lan sudo[21467]: pam_unix(sudo:session): session opened for user root by shane(uid=0)

The first thing I noticed was the Java version, which is 10.0.2+13-Ubuntu… In the past when I worked with Jenkins, JDK was mostly 8. So, I googled about JDK 10+ and Jenkins and found this link. I thought that the content was not convincing me much whether JDK 10+ would work well with Jenkins…

I decided to downgrade to JDK 8 and here’s what I did (did not actual uninstall JDK 10 and reinstall JDK 8)

sudo update-alternatives –config java
There are 2 choices for the alternative java (providing /usr/bin/java).

Selection Path Priority Status
————————————————————
* 0 /usr/lib/jvm/java-11-openjdk-amd64/bin/java 1101 auto mode
1 /usr/lib/jvm/java-11-openjdk-amd64/bin/java 1101 manual mode
2 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java 1081 manual mode

Press to keep the current choice[*], or type selection number: 2
update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java to provide /usr/bin/java (java) in manual mode

[20] ? java -version
openjdk version “1.8.0_191”
OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

After confirming java executable points to 1.8 instead of 10/11, I started Jenkins successfully.

[21] ? sudo systemctl start jenkins 
[22] ? ps aux | grep jenkins
jenkins  21545  0.2  0.2  76632  7640 ?        Ss   14:14   0:00 /lib/systemd/systemd --user
jenkins  21546  0.0  0.0 257560  2936 ?        S    14:14   0:00 (sd-pam)
jenkins  21561  0.0  0.0  19096   188 ?        S    14:14   0:00 /usr/bin/daemon --name=jenkins --inherit --env=JENKINS_HOME=/var/lib/jenkins --output=/var/log/jenkins/jenkins.log --pidfile=/var/run/jenkins/jenkins.pid -- /usr/bin/java -Djava.awt.headless=true -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080
jenkins  21562  100  3.0 2953332 114708 ?      Sl   14:14   0:13 /usr/bin/java -Djava.awt.headless=true -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080
shane    21617  0.0  0.0  13136  1088 pts/0    S+   14:14   0:00 grep --color=auto jenkins

No error and Jenkins is up and running!

RancherOS

Recently I got to know about RancherOS, which is an OS that provides pretty much only Docker host environment so that you can launch Docker containers.

My test environment is Mac OS Sierra with VMWare Fusion 8.5. What I did was as follows:

  1. Download ISO file for RancherOS
  2. Launch ISO via VMWare Fusion with 1GB ram and 10GB disk space (That’s normally what I use for linux distribution guest environment)
  3. Once I was on RancherOS terminal, I created cloud-config.yml file with ssh publick key in it. (Refer to the instruction)
  4. I made a note of IP address for the step #5
  5. After installing to the disk, I chose to reboot. From Mac terminal, I ssh’ed to the IP address of the RancherOS guest after RancherOS guest became available.

Pretty straightforward to install RancherOS on my mac.

RancherOS provides its own command line tool called ‘ros’ and this is the tool to configure the system, docker, etc.

Sync up Docker container time with Host’s

WordPress › Error

There has been a critical error on this website.

Learn more about troubleshooting WordPress.