Magento Commerce

Don’t mess your theme files

Today I am explaining, how you can implement 3rd party services such as Google Analytics, Facebook Pixel or other tracking services in Magento. The reason why I am doing it is because for some reason it is still popular to put JavaScript code directly in theme files, such as page.phtml, success.phtml or footer.phtml. It might be a quick solution, but in most cases very difficult to maintain. With Magento you can easily create blocks which allows you to place the tracking codes by using layout handles without modifying any theme file.

Here is one example that you can easily adapt for other purposes.

1. Create a custom .phtml file

Go to your template folder ./app/design/frontend/{package}/default/template/ and create a new folder custom. Inside the custom folder create a file my_code.phtml and put all your JavaScript code in there.

<script type="text/javascript">
    ...
</script>

...

</script>

2. Create or edit your local.xml file

Create or edit your existing local.xml file which should be located in ./app/design/frontend/{package}/default/layout/local.xml.

<?xml version="1.0"?>
<layout version="0.1.0">
    <default>
        ...
    </default>
</layout>

<?xml version="1.0"?>

...

</default>

</layout>

Inside of the <default> handle you have to place the following <reference>.

<?xml version="1.0"?>
<layout version="0.1.0">
    <default>
        <reference name="after_body_start">
            <block type="core/template" name="my_code" template="custom/my_code.phtml" />
        </reference>
    </default>
</layout>

<?xml version="1.0"?>

</reference>

</default>

</layout>

With the attribute name before_body_end and after_body_start you can define the position inside of the <body>. The <default> layout handle is present in every page request and will add your JavaScript code on all pages. You can use other layout handles such as <checkout_onepage_success> to add your JavaScript code only on the checkout success page.

That’s it. If you clear the cache, you should see the content of my_code.phtml in your HTML code.

3. Pass data to your block

You may need some custom data in your block, such as Google Analytics Account ID or other values for a proper tracking. Those data, you can add to your block with setData.

<block type="core/template" name="my_code" template="custom/my_code.phtml">
    <action method="setData">
        <key>account_id</key>
        <value>1234567</value>
    </action>
    <action method="setData">
        <key>track</key>
        <value>CheckoutSuccess</value>
    </action>
</block>

<key>account_id</key>

</action>

<key>track</key>

<value>CheckoutSuccess</value>

</action>

</block>

Use getData to receive the values.

<script>
    <?php echo $this->getData('account_id'); ?>
    <?php echo $this->getData('track'); ?>
</script>

<?php echo $this->getData('account_id'); ?>

<?php echo $this->getData('track'); ?>

</script>

Protect your AJAX controller action from malicious calls

If you are working with AJAX calls, you should at least verify where the calls are coming from.

if (strpos(Mage::helper('core/http')->getHttpReferer(), Mage::getBaseUrl()) === false)
{
    exit();
}

if (strpos(Mage::helper('core/http')->getHttpReferer(), Mage::getBaseUrl()) === false)

{

exit();

}

Please find the complete code on Gist.

Notice: Array to string conversion in grid.phtml on line 161

I am working on a new module and just received the following notice message when I was trying to open the admin html grid.

Notice: Array to string conversion  in /var/www/html/{VirtualHost}/app/design/adminhtml/default/default/template/widget/grid.phtml on line 161

1	Notice: Array to string conversion in /var/www/html/{VirtualHost}/app/design/adminhtml/default/default/template/widget/grid.phtml on line 161

The reason of this message is that I just copied the Grid.php from another entity that contains different database columns. So I just removed columns which are not defined in install-0.0.1.php ( especially columns with a custom renderer ) to fix the problem.

Cannot create table column without comments

Today I forgot to set a comment for my new column which broke the complete module setup procedure. Here is one example how you can add a new column to the sales flat table ” sales_flat_order_item “.

<?php

$installer = $this;
$installer->startSetup();
$installer->getConnection()->addColumn($installer->getTable('sales/order_item'), 'custom_price_field', array('type' => Varien_Db_Ddl_Table::TYPE_DECIMAL, 'length' => '12,4', 'comment' => 'My new custom price field'));
$installer->endSetup();

?>

<?php

$installer = $this;

$installer->startSetup();

$installer->getConnection()->addColumn($installer->getTable('sales/order_item'), 'custom_price_field', array('type' => Varien_Db_Ddl_Table::TYPE_DECIMAL, 'length' => '12,4', 'comment' => 'My new custom price field'));

$installer->endSetup();

Check Multiple Magento Instances with checkr

This is a quick example how you can check the visibility of specific paths of multiple Magento instances with a shell script called checkr. For example:

/var/log/system.log
/var/log/exception.log
/downloader/
/var/
/backup/
/RELEASE_NOTES.txt

For security reasons, those folders or files should never visible or accessible from outside.

1. URLs

Create a simple list with your clients’ names and its URLs. Save this file as clients.list.

Cafe;http://www.cafe.com/
Lunch;http://www.lunch.com/
Pizza;http://www.pizza.de/

Cafe;http://www.cafe.com/

Lunch;http://www.lunch.com/

Pizza;http://www.pizza.de/

2. Shell Script

#!/bin/sh

# =========================================================================
# Title             checkr.sh
# Description       Check the http response code of specific requests
# Author            Tobias Forkel
# Date              2015/07/10
# Version           0.1
# Usage             ./checkr.sh "/var/log/exception.log"
# Notes
# =========================================================================

if [[ -z $1 ]]; then
echo 'Missing parameter. Example: ./checkr.sh "/var/log/system.log"'
exit
fi

printf '%-20s %s\t %-20s %s\t %-20s %s\n' 'Name' 'Result' 'URL'

while read ROW; do

IFS=';' eval 'ROW=($ROW)'
NAME=${ROW[0]}
URL=${ROW[1]}

# Set the request URL
REQUEST=$(printf '%s%s' $URL $1)

# Receive the http code from curl
RESULT=$(curl -o /dev/null --silent --head --write-out '%{http_code}' $REQUEST)

# Output a formatted row
printf '%-20s %s\t %-20s %s\t %-20s %s\n' $NAME $RESULT $REQUEST

done < clients.list

#!/bin/sh

# =========================================================================

# Title checkr.sh

# Description Check the http response code of specific requests

# Author Tobias Forkel

# Date 2015/07/10

# Version 0.1

# Usage ./checkr.sh "/var/log/exception.log"

# Notes

# =========================================================================

if [[ -z $1 ]]; then

echo 'Missing parameter. Example: ./checkr.sh "/var/log/system.log"'

exit

printf '%-20s %s\t %-20s %s\t %-20s %s\n' 'Name' 'Result' 'URL'

while read ROW; do

IFS=';' eval 'ROW=($ROW)'

NAME=${ROW[0]}

URL=${ROW[1]}

# Set the request URL

REQUEST=$(printf '%s%s' $URL $1)

# Receive the http code from curl

RESULT=$(curl -o /dev/null --silent --head --write-out '%{http_code}' $REQUEST)

# Output a formatted row

printf '%-20s %s\t %-20s %s\t %-20s %s\n' $NAME $RESULT $REQUEST

done < clients.list

3. Run

Run this script with a custom parameter.

./checkr.sh "/var/log/system.log"

1	./checkr.sh "/var/log/system.log"

4. Results

Name                 Result	 URL                  	                   
Magento Commerce     404	 http://www.magentocommerce.com/var/log/system.log
Cafe                 301	 http://www.cafe.com/var/log/system.log
Lunch                503	 http://www.lunch.com/var/log/system.log

Name Result URL

Magento Commerce 404 http://www.magentocommerce.com/var/log/system.log

Cafe 301 http://www.cafe.com/var/log/system.log

Lunch 503 http://www.lunch.com/var/log/system.log

Find Malware in Image Files

It’s a nightmare. Your production environment was compromised and you actually don’t know how and how much data was stolen. For sure there are different ways to be compromised. In this post I just want to explain, how hackers can get full access to your production environment by using image files.

As explained on Snapfast and Sucuri it is pretty easy to store any kind of PHP code in EXIF headers. Often times it only needs a simple script to create an administrator account. If you are using third-party extensions with an image upload function or if you are late with the last security update, there is a high risk to be compromised.

You can use commands like svn status or git status to find changes, but this is no guarantee to find malicious code, because oftentimes your /media/ or similar folders are not version controlled.

Here is one simple command how you can find infected image files.

$ cd /var/www/{{{MAGENTO}}}/
$ find . -iregex '.*\.\(jpg\|gif\)' -exec grep -o -P 'base64_decode' {} \;

1 2	$ cd /var/www/{{{MAGENTO}}}/ $ find . -iregex '.*\.$jpg\\|gif$' -exec grep -o -P 'base64_decode' {} \;

In case if any file is infected, you will see the following search result.

Binary file ./media/test.jpg matches

1	Binary file ./media/test.jpg matches

5 useful commands for your daily work with Magento

1. Backup your media/catalog

Create a manually backup of media/catalog without /cache/ folder

$ cd /var/www/{{{MAGENTO}}}/media/
$ tar -czf ~/catalog.tar.gz catalog --exclude 'catalog/product/cache/*'

1 2	$ cd /var/www/{{{MAGENTO}}}/media/ $ tar -czf ~/catalog.tar.gz catalog --exclude 'catalog/product/cache/*'

The same with the current date in the filename.

$ tar -czf ~/"catalog_$(date '+%Y%m%d').tar.gz" catalog --exclude 'catalog/product/cache/*'

1	$ tar -czf ~/"catalog_$(date '+%Y%m%d').tar.gz" catalog --exclude 'catalog/product/cache/*'

2. Files not owned by apache

Find files which are not owned by apache group or user. This is useful if you want detect possible permission issues.

$ cd /var/www/{{{MAGENTO}}}/
$ find . \! -group www-data \! -user www-data

1 2	$ cd /var/www/{{{MAGENTO}}}/ $ find . \! -group www-data \! -user www-data

3. SQL / CSV files in your document root

You should never left database dump or other export files with sensible information in your document root, because there is a potential risk that those files can be downloaded from outside, depending on your security settings. To make sure your production environment is save, just run a simple ” find ” with the option ” iregex “.

$ cd /var/www/{{{MAGENTO}}}/
$ find -iregex '.*\.\(sql\|tar\|gz\|zip\|csv\)'

1 2	$ cd /var/www/{{{MAGENTO}}}/ $ find -iregex '.*\.$sql\\|tar\\|gz\\|zip\\|csv$'

4. List report files created in the last 24 hours

$ cd /var/www/{{{MAGENTO}}}/var/report/
$ find . -type f -ctime -1

1 2	$ cd /var/www/{{{MAGENTO}}}/var/report/ $ find . -type f -ctime -1

Here are some more options you can use for ” -ctime “.

# find files created greater than 24 hours ago
find . -type f -ctime +0

# find files created between now and 1 day ago
# (i.e., in the past 24 hours only)
find . -type f -ctime 0

# find files created less than 1 day ago
find . -type f -ctime -1

# find files created between 24 and 48 hours ago
find . -type f -ctime 1

# find files created more than 48 hours ago
find . -type f -ctime +1

# find files created greater than 24 hours ago

find . -type f -ctime +0

# find files created between now and 1 day ago

# (i.e., in the past 24 hours only)

find . -type f -ctime 0

# find files created less than 1 day ago

find . -type f -ctime -1

# find files created between 24 and 48 hours ago

find . -type f -ctime 1

# find files created more than 48 hours ago

find . -type f -ctime +1

5. Truncate log files

A fast growing system.log or exception.log can be the cause of a server outage. If you want empty log files on a production environment, the command ” truncate ” will help you.

$ cd /var/www/{{{MAGENTO}}}/var/log
$ truncate -s 0 system.log

1 2	$ cd /var/www/{{{MAGENTO}}}/var/log $ truncate -s 0 system.log

For further debugging you can keep log data by specifying the size.

$ truncate -s 10M system.log

1	$ truncate -s 10M system.log

Practical grep command examples for your Magento log file audit

Improve your bug fixing routine with some simple grep commands. Run these commands before any production deployment just to make sure your new code is working properly. To reduce the output on your terminal, extend my examples as you can see below.

Count the results with ” wc -l ” at the end.

$ grep -oi 'undefined index' system.log | wc -l

1	$ grep -oi 'undefined index' system.log \| wc -l

Display the 1000 rows with ” tail -n 1000 ” at the beginning.

$ tail -n 1000 system.log | grep -i 'undefined index'

1	$ tail -n 1000 system.log \| grep -i 'undefined index'

1. System.log

1.1 Access to an array by a key that doesn’t exist

$ grep -i 'undefined index' system.log

1	$ grep -i 'undefined index' system.log

1.2 Use a variable that wasn’t previously defined

$ grep -i 'undefined variable' system.log

1	$ grep -i 'undefined variable' system.log

1.3 Debug output generated by Mage::log()

$ grep -i 'debug' system.log

1	$ grep -i 'debug' system.log

1.4 Any kind of permission issues

$ grep -i 'permission denied' system.log

1	$ grep -i 'permission denied' system.log

1.5 Missing Magento template files

$ grep -i 'not valid template file' system.log

1	$ grep -i 'not valid template file' system.log

1.6 Access to a non-object

$ grep -i 'trying to get property of non-object' system.log

1	$ grep -i 'trying to get property of non-object' system.log

1.7 Missing argument in a function call

$ grep -i 'missing argument' system.log

1	$ grep -i 'missing argument' system.log

1.8 Invalid argument supplied to a loop

$ grep -i 'invalid argument supplied' system.log

1	$ grep -i 'invalid argument supplied' system.log

Find all:

$ cat system.log | egrep -i '(debug|notice: undefined|permission denied|not valid template file|trying to get property of non-object|missing argument|invalid argument supplied)'

1	$ cat system.log \| egrep -i '(debug\|notice: undefined\|permission denied\|not valid template file\|trying to get property of non-object\|missing argument\|invalid argument supplied)'

2. Exception.log

2.1 Any kind if XML issues

$ grep -i 'xml schema' exception.log

1	$ grep -i 'xml schema' exception.log

2.2 Specific Authorize.net gateway issues

$ grep -i 'paygate.*gateway' exception.log

1	$ grep -i 'paygate.*gateway' exception.log

2.3 Specific PayPal gateway issues

$ grep -i 'paypal.*gateway' exception.log

1	$ grep -i 'paypal.*gateway' exception.log

2.4 Find general curl issues

$ grep -i 'curl' exception.log

1	$ grep -i 'curl' exception.log

Find all:

$ cat exception.log | egrep -i '(xml schema|paygate.*gateway|paypal.*gateway|curl)'

1	$ cat exception.log \| egrep -i '(xml schema\|paygate.gateway\|paypal.gateway\|curl)'

3. Reports

Find the last 10 error report files and grep for any kind of error message.
3.1 MySQL connection issues

$ ls -1t | head -10 | xargs grep -i 'mysql server' -sl

1	$ ls -1t \| head -10 \| xargs grep -i 'mysql server' -sl

3.2 Invalid config field

$ ls -1t | head -10 | xargs grep -i 'invalid config field' -sl

1	$ ls -1t \| head -10 \| xargs grep -i 'invalid config field' -sl

3.3 Unable to read response

$ ls -1t | head -10 | xargs grep -i 'unable to read response' -sl

1	$ ls -1t \| head -10 \| xargs grep -i 'unable to read response' -sl

3.4 External urls redirect

$ ls -1t | head -10 | xargs grep -i 'external urls redirect' -sl

1	$ ls -1t \| head -10 \| xargs grep -i 'external urls redirect' -sl

Find all:

ls -1t | head -10 | xargs  egrep -i '(mysql server|invalid config field|unable to read response|external urls redirect)' -sl

1	ls -1t \| head -10 \| xargs egrep -i '(mysql server\|invalid config field\|unable to read response\|external urls redirect)' -sl

4. Apache error logs

4.1 Find missing files caused by any reason

$ tail -n 1000 /var/log/httpd/{virtualhost}-error_log | grep -i 'file does not exist'

1	$ tail -n 1000 /var/log/httpd/{virtualhost}-error_log \| grep -i 'file does not exist'

Setup cron job for apache user

As long as you don’t need root permissions in your PHP application / script, it is recommended to set up cron jobs with a web server specific user.

1. Find the apache user

Get the current apache user from the process status.

$ ps axo user,group,comm | egrep '(apache|httpd)'
root     root     apache2
www-data www-data apache2

$ ps axo user,group,comm | egrep '(apache|httpd)'

root root apache2

www-data www-data apache2

2. Edit the tab

Edit the crontab with the option -u and the specific apache user. This command will create a user-specific cron file in /var/spool/cron/crontabs/.

$ crontab -u www-data -e

1	$ crontab -u www-data -e

Set up your crontab and save the changes with CTRL-O.

# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
#
# For more information see the manual pages of crontab(5) and cron(8)
#
# m h  dom mon dow   command
*/5 * * * * /bin/sh /var/www/magento/cron.sh

# For example, you can run a backup of all your user accounts

# at 5 a.m every week with:

# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/

# For more information see the manual pages of crontab(5) and cron(8)

# m h dom mon dow command

*/5 * * * * /bin/sh /var/www/magento/cron.sh

3. Check the syslog file

After that, check the syslog file to make sure that the cronjob is running correctly.

$ grep 'CRON'.*\(www-data\) /var/log/syslog

1	$ grep 'CRON'.*$www-data$ /var/log/syslog

Query to get size of database tables in MB

If your hosting environment has no database monitoring, you should be able to monitor it manually once in a while or with your own shell script to prevent rapidly growing MySQL tables, because the result of large MySQL tables ( I am talking about 2 – 10 GB with millions of rows ) are often random performance issues, especially if you are using community modules with not well written MySQL queries.

SELECT TABLE_NAME, table_rows, data_length, index_length,
round(((data_length + index_length) / 1024 / 1024),2) "Size in MB"
FROM information_schema.TABLES WHERE table_schema = "{{{YOUR_DATABASE}}}"
ORDER BY (data_length + index_length) DESC;

SELECT TABLE_NAME, table_rows, data_length, index_length,

round(((data_length + index_length) / 1024 / 1024),2) "Size in MB"

FROM information_schema.TABLES WHERE table_schema = "{{{YOUR_DATABASE}}}"

ORDER BY (data_length + index_length) DESC;

This is the output of the query.

| enterprise_logging_event                      |     122389 |    15939572 |      3270656 |      18.32 |
| catalog_product_index_eav                     |     324014 |     4212182 |     18053120 |      21.23 |
| log_visitor_info                              |     182838 |    19706548 |      3599360 |      22.23 |
| catalog_product_index_eav_idx                 |     324014 |     4522245 |     21139456 |      24.47 |
| catalog_product_entity_varchar                |     255303 |    10852180 |     15244288 |      24.89 |
| log_url                                       |     228886 |    10966704 |     15636480 |      25.37 |
| log_url_info                                  |     228886 |    80258352 |      7542784 |      83.73 |
| enterprise_logging_event_changes              |      69184 |   160619752 |      1499136 |     154.61 |
+-----------------------------------------------+------------+-------------+--------------+------------+
586 rows in set (0.05 sec)

| enterprise_logging_event | 122389 | 15939572 | 3270656 | 18.32 |

| catalog_product_index_eav | 324014 | 4212182 | 18053120 | 21.23 |

| log_visitor_info | 182838 | 19706548 | 3599360 | 22.23 |

| catalog_product_index_eav_idx | 324014 | 4522245 | 21139456 | 24.47 |

| catalog_product_entity_varchar | 255303 | 10852180 | 15244288 | 24.89 |

| log_url | 228886 | 10966704 | 15636480 | 25.37 |

| log_url_info | 228886 | 80258352 | 7542784 | 83.73 |

| enterprise_logging_event_changes | 69184 | 160619752 | 1499136 | 154.61 |

+-----------------------------------------------+------------+-------------+--------------+------------+

586 rows in set (0.05 sec)