Troubleshooting

Use TIMESTAMPDIFF to identify long running cronjobs in Magento2

Improving performance in cron jobs can be very time consuming. Even just figuring out where to start can take hours.

I came up with an idea that really helped me to locate potential performance killer just by checking the execution time by using TIMESTAMPDIFF. For example:

SELECT schedule_id, job_code, executed_at, finished_at, TIMESTAMPDIFF(SECOND, executed_at, finished_at) AS runtime
FROM cron_schedule
ORDER BY runtime desc
LIMIT 10;

SELECT schedule_id, job_code, executed_at, finished_at, TIMESTAMPDIFF(SECOND, executed_at, finished_at) AS runtime

FROM cron_schedule

ORDER BY runtime desc

LIMIT 10;

Output example:

+-------------+------------------------------------+---------------------+---------------------+---------+
| schedule_id | job_code                           | executed_at         | finished_at         | runtime |
+-------------+------------------------------------+---------------------+---------------------+---------+
|    25432321 | custom_customer_sync               | 2022-09-13 18:50:07 | 2022-09-13 18:57:28 |     456 |
|    25433099 | indexer_update_all_views           | 2022-09-13 19:05:07 | 2022-09-13 19:05:13 |       6 |
|    25432315 | indexer_update_all_views           | 2022-09-13 18:35:07 | 2022-09-13 18:35:11 |       4 |
|    25432645 | indexer_update_all_views           | 2022-09-13 18:43:06 | 2022-09-13 18:43:09 |       3 |
|    25433048 | indexer_update_all_views           | 2022-09-13 18:59:06 | 2022-09-13 18:59:09 |       3 |
...
+-------------+------------------------------------+---------------------+---------------------+---------+
20 rows in set (0.01 sec)

+-------------+------------------------------------+---------------------+---------------------+---------+

+-------------+------------------------------------+---------------------+---------------------+---------+

| 25432321 | custom_customer_sync | 2022-09-13 18:50:07 | 2022-09-13 18:57:28 | 456 |

| 25433099 | indexer_update_all_views | 2022-09-13 19:05:07 | 2022-09-13 19:05:13 | 6 |

| 25432315 | indexer_update_all_views | 2022-09-13 18:35:07 | 2022-09-13 18:35:11 | 4 |

| 25432645 | indexer_update_all_views | 2022-09-13 18:43:06 | 2022-09-13 18:43:09 | 3 |

| 25433048 | indexer_update_all_views | 2022-09-13 18:59:06 | 2022-09-13 18:59:09 | 3 |

...

+-------------+------------------------------------+---------------------+---------------------+---------+

20 rows in set (0.01 sec)

The runtime can be a sign of high memory or CPU usage, therefore, you should probably look at those cron jobs first.

Magento 2 – How to fix ” The order confirmation email is not sent “

This morning I was working on an email issue ( multi-store ) where customers were able to purchase but didn’t receive the order confirmation email.

As always, I had a look at what has been deployed recently and checked pretty much all logs on the production environment. Unfortunately, I couldn’t find anything.

I started looking at database changes, especially the core_config_data table, and noticed that the template for the order confirmation was defined twice. This was kind of weird because those email settings were set to ” Use System Value “.

select * from core_config_data where path like 'sales_email/order%' order by scope;
+-----------+----------+----------+------------------------------------------+-------+---------------------+
| config_id | scope    | scope_id | path                                     | value | updated_at          |
+-----------+----------+----------+------------------------------------------+-------+---------------------+
|      1907 | default  |        0 | sales_email/order/template               | 27    | 2021-01-20 04:30:44 |
|      1908 | default  |        0 | sales_email/order/guest_template         | 27    | 2021-01-20 04:30:44 |
|      1909 | default  |        0 | sales_email/order/copy_to                | NULL  | 2021-04-06 07:33:12 ||
|      4293 | stores   |       21 | sales_email/order/template               | 27    | 2021-03-18 05:09:26 |
|      4296 | stores   |       21 | sales_email/order/guest_template         | 27    | 2021-03-18 05:09:26 |
|      4299 | stores   |       21 | sales_email/order/copy_to                | NULL  | 2021-03-20 05:12:37 |
+-----------+----------+----------+------------------------------------------+-------+---------------------+
6 rows in set (0.01 sec)

select * from core_config_data where path like 'sales_email/order%' order by scope;

+-----------+----------+----------+------------------------------------------+-------+---------------------+

+-----------+----------+----------+------------------------------------------+-------+---------------------+

| 1907 | default | 0 | sales_email/order/template | 27 | 2021-01-20 04:30:44 |

| 1908 | default | 0 | sales_email/order/guest_template | 27 | 2021-01-20 04:30:44 |

| 1909 | default | 0 | sales_email/order/copy_to | NULL | 2021-04-06 07:33:12 ||

| 4293 | stores | 21 | sales_email/order/template | 27 | 2021-03-18 05:09:26 |

| 4296 | stores | 21 | sales_email/order/guest_template | 27 | 2021-03-18 05:09:26 |

| 4299 | stores | 21 | sales_email/order/copy_to | NULL | 2021-03-20 05:12:37 |

+-----------+----------+----------+------------------------------------------+-------+---------------------+

6 rows in set (0.01 sec)

I ended up deleting the IDs 4293, 4296, 4299 which has fixed the issue immediately. I still don’t know why those values were set twice, but I hope this will help someone…sometime.

Magento 2.4.x, Elastic Search 7.9.1 on Alpine Linux 3.12, jdk/bin/java: No such file or directory

While setting up a new local test environment specifically for Magento 2.4.x with Elastic Search on Alpine Linux ( Docker ) I had some problems with getting elastic search up and running.

# Use Alpine Linux
FROM alpine:3.12

# Other stuff

ENV ELASTICSEARCH_VERSION 7.9.1

RUN mkdir -p /opt/ && \
  cd /opt && \
  curl -Ls -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-$ELASTICSEARCH_VERSION-linux-x86_64.tar.gz && \
  tar -xvf elasticsearch-$ELASTICSEARCH_VERSION-linux-x86_64.tar.gz && \
  cd elasticsearch-$ELASTICSEARCH_VERSION/bin

# Use Alpine Linux

FROM alpine:3.12

# Other stuff

ENV ELASTICSEARCH_VERSION 7.9.1

RUN mkdir -p /opt/ && \

cd /opt && \

curl -Ls -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-$ELASTICSEARCH_VERSION-linux-x86_64.tar.gz && \

tar -xvf elasticsearch-$ELASTICSEARCH_VERSION-linux-x86_64.tar.gz && \

cd elasticsearch-$ELASTICSEARCH_VERSION/bin

The first issue I had was related to a missing home path of JAVA. Because of that, I wasn’t able to start Elastic Search at all.

bash-5.0# ./elasticsearch
./elasticsearch-env: line 77: /opt/elasticsearch-7.9.1/jdk/bin/java: No such file or directory

1 2	bash-5.0# ./elasticsearch ./elasticsearch-env: line 77: /opt/elasticsearch-7.9.1/jdk/bin/java: No such file or directory

I’ve fixed it by setting the missing path for JAVA_HOME which I have added later to my profile settings. https://stackoverflow.com/questions/35325856/where-to-set-system-default-environment-variables-in-alpine-linux

bash-5.0# which java
/usr/bin/java

1 2	bash-5.0# which java /usr/bin/java

export JAVA_HOME=/usr

1	export JAVA_HOME=/usr

The second issue was related to the JAVA version inside the docker container.

bash-5.0# ./elasticsearch
future versions of Elasticsearch will require Java 11; your Java version from [/usr/lib/jvm/java-1.8-openjdk/jre] does not meet this requirement

1 2	bash-5.0# ./elasticsearch future versions of Elasticsearch will require Java 11; your Java version from [/usr/lib/jvm/java-1.8-openjdk/jre] does not meet this requirement

Alpine Linux has the required version in the repository, which made it easy to install the missing package with apk.

bash-5.0# apk add openjdk11-jre
(1/2) Installing openjdk11-jre-headless (11.0.8_p10-r0)
(2/2) Installing openjdk11-jre (11.0.8_p10-r0)
Executing java-common-0.2-r0.trigger
OK: 649 MiB in 157 packages

bash-5.0# apk add openjdk11-jre

(1/2) Installing openjdk11-jre-headless (11.0.8_p10-r0)

(2/2) Installing openjdk11-jre (11.0.8_p10-r0)

Executing java-common-0.2-r0.trigger

OK: 649 MiB in 157 packages

AWS – ECS Keeps Recreating Docker Container

Today was a strange day. I’ve launched a new version of my docker image within an ECS cluster, which usually worked fine. However, today my container always got recreated every 5 minutes for no reason.

The new docker image that I have pushed to the repository had no significant changes, except a redirect from non-SSL ( 80 ) to SSL ( 443 ) for all loaded virtual hosts. So, when I had a look at the processes, everything looked fine. There was nothing that could put the container into an unhealthy status.

PID   USER     TIME  COMMAND
    1 root      0:02 {supervisord} /usr/bin/python2 /usr/bin/supervisord -c /etc/supervisord.conf
    8 root      0:00 nginx: master process /usr/sbin/nginx -g daemon off;
    9 root      0:00 {mysqld.sh} /bin/sh /scripts/mysqld.sh
   10 root      0:00 /usr/sbin/crond -f
   13 nginx     0:13 nginx: worker process
   35 root      0:00 {php-fpm7.1} php-fpm: master process (/etc/php/7.1/php-fpm.conf)
   90 root      0:00 {mysqld_safe} /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql
  288 mysql     0:08 /usr/bin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mariadb/plugin --user=mysql --log-error=/var/lib/mysql/f817894c1752.err --pid-file=f817894c1752.pid --socket=/var/run/my
 3620 www-data  0:06 {php-fpm7.1} php-fpm: pool www
 3621 www-data  0:05 {php-fpm7.1} php-fpm: pool www
 3668 www-data  0:02 {php-fpm7.1} php-fpm: pool www
 3795 root      0:00 bash
 3807 root      0:00 ps aux

PID USER TIME COMMAND

1 root 0:02 {supervisord} /usr/bin/python2 /usr/bin/supervisord -c /etc/supervisord.conf

8 root 0:00 nginx: master process /usr/sbin/nginx -g daemon off;

9 root 0:00 {mysqld.sh} /bin/sh /scripts/mysqld.sh

10 root 0:00 /usr/sbin/crond -f

13 nginx 0:13 nginx: worker process

35 root 0:00 {php-fpm7.1} php-fpm: master process (/etc/php/7.1/php-fpm.conf)

90 root 0:00 {mysqld_safe} /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql

288 mysql 0:08 /usr/bin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mariadb/plugin --user=mysql --log-error=/var/lib/mysql/f817894c1752.err --pid-file=f817894c1752.pid --socket=/var/run/my

3620 www-data 0:06 {php-fpm7.1} php-fpm: pool www

3621 www-data 0:05 {php-fpm7.1} php-fpm: pool www

3668 www-data 0:02 {php-fpm7.1} php-fpm: pool www

3795 root 0:00 bash

3807 root 0:00 ps aux

Even the log output of the docker container didn’t show any error. After a little bit of research, I ended up on the following AWS developer guide page https://docs.aws.amazon.com/AmazonECS/latest/developerguide/update-service.html

When the service scheduler replaces a task during an update, the service first removes the task from the load balancer (if used) and waits for the connections to drain. Then, the equivalent of docker stop is issued to the containers running in the task. This results in a SIGTERM signal and a 30-second timeout, after which SIGKILL is sent and the containers are forcibly stopped. If the container handles the SIGTERM signal gracefully and exits within 30 seconds from receiving it, no SIGKILL signal is sent. The service scheduler starts and stops tasks as defined by your minimum healthy percent and maximum percent settings.

When the service scheduler replaces a task during an update, the service first removes the task from the load balancer (if used) and waits for the connections to drain. Then, the equivalent of docker stop is issued to the containers running in the task. This results in a SIGTERM signal and a 30-second timeout, after which SIGKILL is sent and the containers are forcibly stopped. If the container handles the SIGTERM signal gracefully and exits within 30 seconds from receiving it, no SIGKILL signal is sent. The service scheduler starts and stops tasks as defined by your minimum healthy percent and maximum percent settings.

It was not exactly what I was looking for, but it pretty much explained the problem I had. One of my colleagues said I should maybe have a look at the health checks of the load balancer.

Indeed, when I had a look at the settings in EC2 > Load Balancing > Target Groups > Health Checks the health check expected a success code 200 instead of 301 on port 80. Because of that, the container has been marked unhealthy every 5 minutes.

I hope this will point someone in the right direction.

Octopus – How to fix ” There must be at least one enabled healthy machine to deploy “

I am pretty new to Octopus and just started implementing a deployment process for a new PHP project. After creating a new machine in Infrastructure > Deployment Targets I went back to Projects > My Project > Create Release and tried to start a deployment to our dev environment which caused the following error.

There must be at least one enabled healthy machine to deploy to in the environment, or the Project Deployment Target settings must be set to allow deployments to empty environments. Once you have corrected these problems you can try again. If the problem is related to a variable you will need to update the variables for this release or recreate the release for the changes to take effect. If the problem is related to the deployment process you will need to create a new release for the changes to take effect.

There must be at least one enabled healthy machine to deploy to in the environment, or the Project Deployment Target settings must be set to allow deployments to empty environments. Once you have corrected these problems you can try again. If the problem is related to a variable you will need to update the variables for this release or recreate the release for the changes to take effect. If the problem is related to the deployment process you will need to create a new release for the changes to take effect.

As you can see in the below screenshot, I tried to deploy a new release to the target Development.

The error message was strange because the target status of my Deployment Target was flagged as healthy. After comparing the settings I noticed that the environment Development in my deployment target settings was simply missing.

After adding the target Development to the list of Environments, I was able to proceed with the above deployment.

October CMS – Widget OC Media Manager

If you work with the form elements fileupload or mediafinder in combination with user admin roles ( Settings > Administrators ) you may end up with the following error while saving a model.

"A widget with class name 'ocmediamanager' has not been bound to the controller" on line 597 of /var/www/src/modules/backend/classes/Controller.php

1	"A widget with class name 'ocmediamanager' has not been bound to the controller" on line 597 of /var/www/src/modules/backend/classes/Controller.php

In order to fix that, you must review the permissions of your admin user account for Upload and manage media contents – images, videos, sounds, documents. and select Allow or Inherit, depending on how you setup your roles.

After saving the permissions, you should be able to upload or select an image and save the model.

October CMS – How To Fix Broken Thumbnails

I recently started setting up a dockerized October CMS environment based on Alpine Linux, MariaDB and PHP-FPM which was actually a very straightforward process. However, for some reason the integrated image resizer didn’t create thumbnails of uploaded images.

Unfortunately there was nothing in the log files which could help me to identify the cause of the problem. After checking the file permissions, I decided to trace back the issue and found the reason in file vendor/october/rain/src/Database/Attach/Resizer.php.

The method getMimeType() returned always null, which broke the entire image manipulation process.

// Get the file extension
$this->extension = $file->guessExtension();
$this->mime = $file->getMimeType();

// Get the file extension

$this->extension = $file->guessExtension();

$this->mime = $file->getMimeType();

After a quick research I found out that getMimeType requires php_fileinfo which I simply forgot to set in my Dockerfile. After adding php7-fileinfo and rebuilding the container, the issue was finally fixed.

Notice: Use of undefined constant T_CURLY_OPEN – assumed ‘T_CURLY_OPEN’ in /var/www/src/setup/src/ Magento/Setup/Module/ Di/Code/Reader/FileClassScanner.php on line 72

Today I finally had some time to cleaned up my Dockerfile for Magento 2. I am using Alpine Linux for most of my Magento 2 projects which usually includes NGINX, PHP-FPM and MariaDB.

# Use Alpine Linux
FROM alpine:3.7

# Declare maintainer
MAINTAINER Tobias Forkel <tobias@forkel.me>

# Timezone
ENV TIMEZONE Australia/Melbourne

RUN addgroup -g 1000 www && \
    adduser -S www -G www -s /sbin/nologin -D

ADD https://repos.php.earth/alpine/phpearth.rsa.pub /etc/apk/keys/phpearth.rsa.pub
RUN echo "https://repos.php.earth/alpine/v3.7" >> /etc/apk/repositories

RUN apk --no-cache add \
    bash \
    git \
	openssl \
	nginx \
	supervisor \
	curl \
	mariadb \
	mariadb-client \
	libxml2-utils \
	php7.1-common \
	php7.1-fpm \
	php7.1-json \
....

# Use Alpine Linux

FROM alpine:3.7

# Declare maintainer

MAINTAINER Tobias Forkel <tobias@forkel.me>

# Timezone

ENV TIMEZONE Australia/Melbourne

RUN addgroup -g 1000 www && \

adduser -S www -G www -s /sbin/nologin -D

ADD https://repos.php.earth/alpine/phpearth.rsa.pub /etc/apk/keys/phpearth.rsa.pub

RUN echo "https://repos.php.earth/alpine/v3.7" >> /etc/apk/repositories

RUN apk --no-cache add \

bash \

git \

openssl \

nginx \

supervisor \

curl \

mariadb \

mariadb-client \

libxml2-utils \

php7.1-common \

php7.1-fpm \

php7.1-json \

....

While testing my updated Dockerfile I came across the following PHP notice which prevented the deployment scripts from finishing the Magento 2 setup.

Notice: Use of undefined constant T_CURLY_OPEN - assumed 'T_CURLY_OPEN' in /var/www/src/setup/src/Magento/Setup/Module/ Di/Code/Reader/FileClassScanner.php on line 72

1	Notice: Use of undefined constant T_CURLY_OPEN - assumed 'T_CURLY_OPEN' in /var/www/src/setup/src/Magento/Setup/Module/ Di/Code/Reader/FileClassScanner.php on line 72

The problem was simply a missing PHP extension php7.1-tokenizer which has resolved the issue after re-creating the container with –build.

docker-compose up -d --build

1	docker-compose up -d --build

PHP message: PHP Fatal error: Interface ‘SessionHandlerInterface’ not found in /var/www/src/vendor/ laravel/framework/src/Illuminate/Session/FileSessionHandler.php on line 10

Today I successfully installed my first October CMS project locally via composer / command line. The installation was pretty straightforward and worked surprisingly well on my Alpine Linux docker container which includes NGINX, PHP-FPM and MariaDB.

October.Demo
- v1.0.1:  First version of Demo
Seeded System 
Seeded Backend 
.=========================================.
                ,@@@@@@@,                  
        ,,,.   ,@@@@@@/@@,  .oo8888o.      
     ,&%%&%&&%,@@@@@/@@@@@@,8888\88/8o     
    ,%&\%&&%&&%,@@@\@@@/@@@88\88888/88'    
    %&&%&%&/%&&%@@\@@/ /@@@88888\88888'    
    %&&%/ %&%%&&@@\ V /@@' `88\8 `/88'     
    `&%\ ` /%&'    |.|        \ '|8'       
        |o|        | |         | |         
        |.|        | |         | |         
`========= INSTALLATION COMPLETE ========='

October.Demo

- v1.0.1: First version of Demo

Seeded System

Seeded Backend

.=========================================.

,@@@@@@@,

,,,. ,@@@@@@/@@, .oo8888o.

,&%%&%&&%,@@@@@/@@@@@@,8888\88/8o

,%&\%&&%&&%,@@@\@@@/@@@88\88888/88'

%&&%&%&/%&&%@@\@@/ /@@@88888\88888'

%&&%/ %&%%&&@@\ V /@@' `88\8 `/88'

`&%\ ` /%&' |.| \ '|8'

|o| | | | |

|.| | | | |

`========= INSTALLATION COMPLETE ========='

However, when I tried to open the front-end I got the following error.

PHP message: PHP Fatal error:  Interface 'SessionHandlerInterface' not found in /var/www/src/vendor/laravel/framework/src/Illuminate/Session/FileSessionHandler.php on line 10

1	PHP message: PHP Fatal error: Interface 'SessionHandlerInterface' not found in /var/www/src/vendor/laravel/framework/src/Illuminate/Session/FileSessionHandler.php on line 10

After a quick research I found the solution on Stackoverflow. I basically forgot to install the PHP extension php7-session inside my docker container. I manually installed the missing extension, but also updated my Dockerfile in case I have to rebuild my container.

bash-4.4# apk add php7-session
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
(1/1) Installing php7-session (7.1.33-r0)
OK: 291 MiB in 87 packages

bash-4.4# apk add php7-session

fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz

fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz

(1/1) Installing php7-session (7.1.33-r0)

OK: 291 MiB in 87 packages

After that I manually killed all PHP processes to make sure php-fpm loads the new extension with the next page reload.

pkill php

pkill php

Font Awesome Icons without CDN in Magento2

I recently worked on a custom module where one of the requirements was to use Font Awesome for custom forms and notifications / alerts.

Most of the people in the Magento community suggest to load Font Awesome via CDN, but this can cause ” blank squares ” and eventually slow down the front-end on slower connections.

<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <head>
        <css src="https://use.fontawesome.com/releases/v5.2.0/css/all.css" src_type="url"/>
    </head>
</page>

<?xml version="1.0"?>

<head>

</head>

</page>

So instead using a CDN I decided to add those libraries in Magento which will be merged with the static view files. Here is how it works.

1. Copy files

Go to Host Font Awesome Yourself and download the entire library. The zip file should contain css, js and less files as well as webfonts.

Copy all less files to the following folder.

[Namespace]/[YourModule]/view/frontend/web/css/fontawesome/

1	[Namespace]/[YourModule]/view/frontend/web/css/fontawesome/

Next, copy all files from the webfonts folder to the below path.

[Namespace]/[YourModule]/view/frontend/web/css/webfonts/

1	[Namespace]/[YourModule]/view/frontend/web/css/webfonts/

Make sure you only copy the less files ( no CSS files ).

2. Update Layout XML

Don’t load the library globally if you only need it at one place. In my case, I added those library to my custom controller only. For example:

[Namespace]/[YourModule]/view/frontend/layout/namespace_yourmodule_controller_action.xml

1	[Namespace]/[YourModule]/view/frontend/layout/namespace_yourmodule_controller_action.xml

<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <head>
        <css src="Namespace_YourModule::css/fontawesome/fontawesome.css"/>
        <css src="Namespace_YourModule::css/fontawesome/regular.css"/>
        <css src="Namespace_YourModule::css/fontawesome/solid.css"/>
        <css src="Namespace_YourModule::css/style.css"/>
    </head>
    <body>
        <referenceContainer name="content">
            ...
        </referenceContainer>
    </body>
</page>

<?xml version="1.0"?>

<head>

</head>

<body>

...

</referenceContainer>

</body>

</page>

3. Template

Once you have re-compiled your code, you should be able to use Front Awesome in your template. For example:

[Namespace]/[YourModule]/view/frontend/templates/form.phtml

1	[Namespace]/[YourModule]/view/frontend/templates/form.phtml

<button class="action-quicklogin">
    <i class="fas fa-sign-in-alt"></i>
</button>

</button>

Btw. the same works for the adminhtml area in ./view/adminhtml/.