Problem

Use TIMESTAMPDIFF to identify long running cronjobs in Magento2

Improving performance in cron jobs can be very time consuming. Even just figuring out where to start can take hours.

I came up with an idea that really helped me to locate potential performance killer just by checking the execution time by using TIMESTAMPDIFF. For example:

SELECT schedule_id, job_code, executed_at, finished_at, TIMESTAMPDIFF(SECOND, executed_at, finished_at) AS runtime
FROM cron_schedule
ORDER BY runtime desc
LIMIT 10;

SELECT schedule_id, job_code, executed_at, finished_at, TIMESTAMPDIFF(SECOND, executed_at, finished_at) AS runtime

FROM cron_schedule

ORDER BY runtime desc

LIMIT 10;

Output example:

+-------------+------------------------------------+---------------------+---------------------+---------+
| schedule_id | job_code                           | executed_at         | finished_at         | runtime |
+-------------+------------------------------------+---------------------+---------------------+---------+
|    25432321 | custom_customer_sync               | 2022-09-13 18:50:07 | 2022-09-13 18:57:28 |     456 |
|    25433099 | indexer_update_all_views           | 2022-09-13 19:05:07 | 2022-09-13 19:05:13 |       6 |
|    25432315 | indexer_update_all_views           | 2022-09-13 18:35:07 | 2022-09-13 18:35:11 |       4 |
|    25432645 | indexer_update_all_views           | 2022-09-13 18:43:06 | 2022-09-13 18:43:09 |       3 |
|    25433048 | indexer_update_all_views           | 2022-09-13 18:59:06 | 2022-09-13 18:59:09 |       3 |
...
+-------------+------------------------------------+---------------------+---------------------+---------+
20 rows in set (0.01 sec)

+-------------+------------------------------------+---------------------+---------------------+---------+

+-------------+------------------------------------+---------------------+---------------------+---------+

| 25432321 | custom_customer_sync | 2022-09-13 18:50:07 | 2022-09-13 18:57:28 | 456 |

| 25433099 | indexer_update_all_views | 2022-09-13 19:05:07 | 2022-09-13 19:05:13 | 6 |

| 25432315 | indexer_update_all_views | 2022-09-13 18:35:07 | 2022-09-13 18:35:11 | 4 |

| 25432645 | indexer_update_all_views | 2022-09-13 18:43:06 | 2022-09-13 18:43:09 | 3 |

| 25433048 | indexer_update_all_views | 2022-09-13 18:59:06 | 2022-09-13 18:59:09 | 3 |

...

+-------------+------------------------------------+---------------------+---------------------+---------+

20 rows in set (0.01 sec)

The runtime can be a sign of high memory or CPU usage, therefore, you should probably look at those cron jobs first.

Magento 2 – How to fix ” The order confirmation email is not sent “

This morning I was working on an email issue ( multi-store ) where customers were able to purchase but didn’t receive the order confirmation email.

As always, I had a look at what has been deployed recently and checked pretty much all logs on the production environment. Unfortunately, I couldn’t find anything.

I started looking at database changes, especially the core_config_data table, and noticed that the template for the order confirmation was defined twice. This was kind of weird because those email settings were set to ” Use System Value “.

select * from core_config_data where path like 'sales_email/order%' order by scope;
+-----------+----------+----------+------------------------------------------+-------+---------------------+
| config_id | scope    | scope_id | path                                     | value | updated_at          |
+-----------+----------+----------+------------------------------------------+-------+---------------------+
|      1907 | default  |        0 | sales_email/order/template               | 27    | 2021-01-20 04:30:44 |
|      1908 | default  |        0 | sales_email/order/guest_template         | 27    | 2021-01-20 04:30:44 |
|      1909 | default  |        0 | sales_email/order/copy_to                | NULL  | 2021-04-06 07:33:12 ||
|      4293 | stores   |       21 | sales_email/order/template               | 27    | 2021-03-18 05:09:26 |
|      4296 | stores   |       21 | sales_email/order/guest_template         | 27    | 2021-03-18 05:09:26 |
|      4299 | stores   |       21 | sales_email/order/copy_to                | NULL  | 2021-03-20 05:12:37 |
+-----------+----------+----------+------------------------------------------+-------+---------------------+
6 rows in set (0.01 sec)

select * from core_config_data where path like 'sales_email/order%' order by scope;

+-----------+----------+----------+------------------------------------------+-------+---------------------+

+-----------+----------+----------+------------------------------------------+-------+---------------------+

| 1907 | default | 0 | sales_email/order/template | 27 | 2021-01-20 04:30:44 |

| 1908 | default | 0 | sales_email/order/guest_template | 27 | 2021-01-20 04:30:44 |

| 1909 | default | 0 | sales_email/order/copy_to | NULL | 2021-04-06 07:33:12 ||

| 4293 | stores | 21 | sales_email/order/template | 27 | 2021-03-18 05:09:26 |

| 4296 | stores | 21 | sales_email/order/guest_template | 27 | 2021-03-18 05:09:26 |

| 4299 | stores | 21 | sales_email/order/copy_to | NULL | 2021-03-20 05:12:37 |

+-----------+----------+----------+------------------------------------------+-------+---------------------+

6 rows in set (0.01 sec)

I ended up deleting the IDs 4293, 4296, 4299 which has fixed the issue immediately. I still don’t know why those values were set twice, but I hope this will help someone…sometime.

October CMS – Widget OC Media Manager

If you work with the form elements fileupload or mediafinder in combination with user admin roles ( Settings > Administrators ) you may end up with the following error while saving a model.

"A widget with class name 'ocmediamanager' has not been bound to the controller" on line 597 of /var/www/src/modules/backend/classes/Controller.php

1	"A widget with class name 'ocmediamanager' has not been bound to the controller" on line 597 of /var/www/src/modules/backend/classes/Controller.php

In order to fix that, you must review the permissions of your admin user account for Upload and manage media contents – images, videos, sounds, documents. and select Allow or Inherit, depending on how you setup your roles.

After saving the permissions, you should be able to upload or select an image and save the model.

Notice: Use of undefined constant T_CURLY_OPEN – assumed ‘T_CURLY_OPEN’ in /var/www/src/setup/src/ Magento/Setup/Module/ Di/Code/Reader/FileClassScanner.php on line 72

Today I finally had some time to cleaned up my Dockerfile for Magento 2. I am using Alpine Linux for most of my Magento 2 projects which usually includes NGINX, PHP-FPM and MariaDB.

# Use Alpine Linux
FROM alpine:3.7

# Declare maintainer
MAINTAINER Tobias Forkel <tobias@forkel.me>

# Timezone
ENV TIMEZONE Australia/Melbourne

RUN addgroup -g 1000 www && \
    adduser -S www -G www -s /sbin/nologin -D

ADD https://repos.php.earth/alpine/phpearth.rsa.pub /etc/apk/keys/phpearth.rsa.pub
RUN echo "https://repos.php.earth/alpine/v3.7" >> /etc/apk/repositories

RUN apk --no-cache add \
    bash \
    git \
	openssl \
	nginx \
	supervisor \
	curl \
	mariadb \
	mariadb-client \
	libxml2-utils \
	php7.1-common \
	php7.1-fpm \
	php7.1-json \
....

# Use Alpine Linux

FROM alpine:3.7

# Declare maintainer

MAINTAINER Tobias Forkel <tobias@forkel.me>

# Timezone

ENV TIMEZONE Australia/Melbourne

RUN addgroup -g 1000 www && \

adduser -S www -G www -s /sbin/nologin -D

ADD https://repos.php.earth/alpine/phpearth.rsa.pub /etc/apk/keys/phpearth.rsa.pub

RUN echo "https://repos.php.earth/alpine/v3.7" >> /etc/apk/repositories

RUN apk --no-cache add \

bash \

git \

openssl \

nginx \

supervisor \

curl \

mariadb \

mariadb-client \

libxml2-utils \

php7.1-common \

php7.1-fpm \

php7.1-json \

....

While testing my updated Dockerfile I came across the following PHP notice which prevented the deployment scripts from finishing the Magento 2 setup.

Notice: Use of undefined constant T_CURLY_OPEN - assumed 'T_CURLY_OPEN' in /var/www/src/setup/src/Magento/Setup/Module/ Di/Code/Reader/FileClassScanner.php on line 72

1	Notice: Use of undefined constant T_CURLY_OPEN - assumed 'T_CURLY_OPEN' in /var/www/src/setup/src/Magento/Setup/Module/ Di/Code/Reader/FileClassScanner.php on line 72

The problem was simply a missing PHP extension php7.1-tokenizer which has resolved the issue after re-creating the container with –build.

docker-compose up -d --build

1	docker-compose up -d --build

PHP message: PHP Fatal error: Interface ‘SessionHandlerInterface’ not found in /var/www/src/vendor/ laravel/framework/src/Illuminate/Session/FileSessionHandler.php on line 10

Today I successfully installed my first October CMS project locally via composer / command line. The installation was pretty straightforward and worked surprisingly well on my Alpine Linux docker container which includes NGINX, PHP-FPM and MariaDB.

October.Demo
- v1.0.1:  First version of Demo
Seeded System 
Seeded Backend 
.=========================================.
                ,@@@@@@@,                  
        ,,,.   ,@@@@@@/@@,  .oo8888o.      
     ,&%%&%&&%,@@@@@/@@@@@@,8888\88/8o     
    ,%&\%&&%&&%,@@@\@@@/@@@88\88888/88'    
    %&&%&%&/%&&%@@\@@/ /@@@88888\88888'    
    %&&%/ %&%%&&@@\ V /@@' `88\8 `/88'     
    `&%\ ` /%&'    |.|        \ '|8'       
        |o|        | |         | |         
        |.|        | |         | |         
`========= INSTALLATION COMPLETE ========='

October.Demo

- v1.0.1: First version of Demo

Seeded System

Seeded Backend

.=========================================.

,@@@@@@@,

,,,. ,@@@@@@/@@, .oo8888o.

,&%%&%&&%,@@@@@/@@@@@@,8888\88/8o

,%&\%&&%&&%,@@@\@@@/@@@88\88888/88'

%&&%&%&/%&&%@@\@@/ /@@@88888\88888'

%&&%/ %&%%&&@@\ V /@@' `88\8 `/88'

`&%\ ` /%&' |.| \ '|8'

|o| | | | |

|.| | | | |

`========= INSTALLATION COMPLETE ========='

However, when I tried to open the front-end I got the following error.

PHP message: PHP Fatal error:  Interface 'SessionHandlerInterface' not found in /var/www/src/vendor/laravel/framework/src/Illuminate/Session/FileSessionHandler.php on line 10

1	PHP message: PHP Fatal error: Interface 'SessionHandlerInterface' not found in /var/www/src/vendor/laravel/framework/src/Illuminate/Session/FileSessionHandler.php on line 10

After a quick research I found the solution on Stackoverflow. I basically forgot to install the PHP extension php7-session inside my docker container. I manually installed the missing extension, but also updated my Dockerfile in case I have to rebuild my container.

bash-4.4# apk add php7-session
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
(1/1) Installing php7-session (7.1.33-r0)
OK: 291 MiB in 87 packages

bash-4.4# apk add php7-session

fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz

fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz

(1/1) Installing php7-session (7.1.33-r0)

OK: 291 MiB in 87 packages

After that I manually killed all PHP processes to make sure php-fpm loads the new extension with the next page reload.

pkill php

pkill php

The use of function filectime() / filemtime() is discouraged in Magento2

The problem I had today was located in a helper ( [Namespace]\[Module]\Helper\FileSystem ) which I have created for some basic filesystem operations. The below message was the result of Magento’s EQP which I use before I commit any code.

 68 | WARNING | The use of function filectime() is discouraged
 69 | WARNING | The use of function filemtime() is discouraged

1 2	68 \| WARNING \| The use of function filectime() is discouraged 69 \| WARNING \| The use of function filemtime() is discouraged

The warning message basically means, I should find another way to get the creation and modification time of a file because of Magento’s coding standards. I have done that a couple of times already and would like to share my solution this time. First you have to pass \Magento\Framework\Filesystem\Driver\File in your existing constructor or create a new one.

protected $driverFile;
...
public function __construct(
    ...
    \Magento\Framework\Filesystem\Driver\File $driverFile,
    ...
) {
    $this->driverFile = $driverFile;
    ...
    parent::__construct($context);
}

protected $driverFile;

...

public function __construct(

...

\Magento\Framework\Filesystem\Driver\File $driverFile,

...

) {

$this->driverFile = $driverFile;

...

parent::__construct($context);

}

Once you have re-compiled your code, you have access to the file stats as follows.

$file = '/absolute/path/to/your/file.extension';
$fileData = $this->driverFile->stat($file);

1 2	$file = '/absolute/path/to/your/file.extension'; $fileData = $this->driverFile->stat($file);

Beside ctime and mtime the array $fileData should have other file information like size, atime, uid and gid.

UI Components and searchResultToOutput in Magento2

Today I had to fix the following issue in a custom module. The error appeared on the UI Component list view.

Fatal error: Method Magento\Ui\TemplateEngine\Xhtml\Result::__toString() must not throw an exception, caught TypeError: Argument 1 passed to Magento\Framework\View\Element\UiComponent\DataProvider\DataProvider::searchResultToOutput() must implement interface Magento\Framework\Api\Search\SearchResultInterface, instance of [Namespace]\[Module]\Model\ResourceModel\Team\Collection given, called in

Fatal error: Method Magento\Ui\TemplateEngine\Xhtml\Result::__toString() must not throw an exception, caught TypeError: Argument 1 passed to Magento\Framework\View\Element\UiComponent\DataProvider\DataProvider::searchResultToOutput() must implement interface Magento\Framework\Api\Search\SearchResultInterface, instance of [Namespace]\[Module]\Model\ResourceModel\Team\Collection given, called in

I found the reason of the problem in the di.xml file. All what was missing was the following virtualType node. After adding this and re-compiling the code, the list view was fixed.

<virtualType name="[Namespace]\[Module]\Model\ResourceModel\Team\Collection" type="Magento\Framework\View\Element\UiComponent\DataProvider\SearchResult">
    <arguments>
        <argument name="mainTable" xsi:type="string">[namespace]_[module]_team</argument>
        <argument name="resourceModel" xsi:type="string">[Namespace]\[Module]\Model\ResourceModel\Team</argument>
    </arguments>
</virtualType>

<argument name="mainTable" xsi:type="string">[namespace]_[module]_team</argument>

<argument name="resourceModel" xsi:type="string">[Namespace]\[Module]\Model\ResourceModel\Team</argument>

</arguments>

</virtualType>

UI Component Listing And Not registered handle in Magento2

Today I created a new listing for a custom module, but unfortunately all what I got was the error message below.

Not registered handle [namespace]_[module]_team_index_data_source

1	Not registered handle [namespace]_[module]_team_index_data_source

After digging around for few minutes I noticed the above handler was actually missing in the di.xml file. I simply added the following collections argument and re-compiled the code.

<type name="Magento\Framework\View\Element\UiComponent\DataProvider\CollectionFactory">
    <arguments>
        <argument name="collections" xsi:type="array">
            ...
            <item name="[namespace]_[module]_team_index_data_source" xsi:type="string">[Namespace]\[Module]\Model\ResourceModel\Team\Collection</item>
            ...
        </argument>
    </arguments>
</type>

...

<item name="[namespace]_[module]_team_index_data_source" xsi:type="string">[Namespace]\[Module]\Model\ResourceModel\Team\Collection</item>

...

</argument>

</arguments>

</type>

All good!

Missing required argument $name of Xtento\ProductExport\Logger\Logger.

If you are using Xtento in combination with an automated product importer, you may have seen the following error before.

Missing required argument $name of Xtento\ProductExport\Logger\Logger.

1	Missing required argument $name of Xtento\ProductExport\Logger\Logger.

The error message appears right after saving a product. The reason is that, at least one of your products in Products > Catalog has no Name. To fix this, you can update the name manually via MySQL queries or simply delete the products and import again correctly.

Category “Default Category/abc/def/” has not been created. URL key for specified store already exists in Magento2

A new day, a new problem. While importing new products in Magento 2.1.2 via System > Data Transfer > Import something was causing the following error message.

1. Category "Default Category/abc/def/" has not been created. URL key for specified store already exists. in row(s): 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

1	1. Category "Default Category/abc/def/" has not been created. URL key for specified store already exists. in row(s): 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

The CSV file was 100% correct, which was making it difficult to figure out why it is causing the error. I ran few more test imports and noticed that the main category actually had an empty space at the end, which I believe was causing the whole problem.

The import ran through without any errors after removing the empty space. However, before I also removed all related URL rewrites as follows. Not sure if this was actually necessary.

mysql> DELETE FROM url_rewrite WHERE entity_type = 'category' AND request_path like '%abc/def/%';
Query OK, 6 rows affected (0.01 sec)

1 2	mysql> DELETE FROM url_rewrite WHERE entity_type = 'category' AND request_path like '%abc/def/%'; Query OK, 6 rows affected (0.01 sec)

After that I rebuilt the re-write table with https://github.com/olegkoval/magento2-regenerate_url_rewrites because for some reason the importer didn’t create the rewrite URLs.

$ php bin/magento ok:urlrewrites:regenerate
Regenerating of Url rewrites:

[Store ID: 0, Store View code: admin]:....................
....................
....................
....................
....................
....................
....................
....................
....................
....................
................URL key for specified store already exists.
....
..

$ php bin/magento ok:urlrewrites:regenerate

Regenerating of Url rewrites:

[Store ID: 0, Store View code: admin]:....................

....................

................URL key for specified store already exists.

....

After that everything was working as expected.