PHP / Magento Developer
If you are working with AJAX calls, you should at least verify where the calls are coming from.
|
1 2 3 4 |
if (strpos(Mage::helper('core/http')->getHttpReferer(), Mage::getBaseUrl()) === false) { exit(); } |
Please find the complete code on Gist.
I am working on a new module and just received the following notice message when I was trying to open the admin html grid.
|
1 |
Notice: Array to string conversion in /var/www/html/{VirtualHost}/app/design/adminhtml/default/default/template/widget/grid.phtml on line 161 |
The reason of this message is that I just copied the Grid.php from another entity that contains different database columns. So I just removed columns which are not defined in install-0.0.1.php ( especially columns with a custom renderer ) to fix the problem.
Today I forgot to set a comment for my new column which broke the complete module setup procedure. Here is one example how you can add a new column to the sales flat table ” sales_flat_order_item “.
|
1 2 3 4 5 6 7 8 |
<?php $installer = $this; $installer->startSetup(); $installer->getConnection()->addColumn($installer->getTable('sales/order_item'), 'custom_price_field', array('type' => Varien_Db_Ddl_Table::TYPE_DECIMAL, 'length' => '12,4', 'comment' => 'My new custom price field')); $installer->endSetup(); ?> |
This is a quick example how you can check the visibility of specific paths of multiple Magento instances with a shell script called checkr. For example:
For security reasons, those folders or files should never visible or accessible from outside.
Create a simple list with your clients’ names and its URLs. Save this file as clients.list.
|
1 2 3 |
Cafe;http://www.cafe.com/ Lunch;http://www.lunch.com/ Pizza;http://www.pizza.de/ |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
#!/bin/sh # ========================================================================= # Title checkr.sh # Description Check the http response code of specific requests # Author Tobias Forkel # Date 2015/07/10 # Version 0.1 # Usage ./checkr.sh "/var/log/exception.log" # Notes # ========================================================================= if [[ -z $1 ]]; then echo 'Missing parameter. Example: ./checkr.sh "/var/log/system.log"' exit fi printf '%-20s %s\t %-20s %s\t %-20s %s\n' 'Name' 'Result' 'URL' while read ROW; do IFS=';' eval 'ROW=($ROW)' NAME=${ROW[0]} URL=${ROW[1]} # Set the request URL REQUEST=$(printf '%s%s' $URL $1) # Receive the http code from curl RESULT=$(curl -o /dev/null --silent --head --write-out '%{http_code}' $REQUEST) # Output a formatted row printf '%-20s %s\t %-20s %s\t %-20s %s\n' $NAME $RESULT $REQUEST done < clients.list |
Run this script with a custom parameter.
|
1 |
./checkr.sh "/var/log/system.log" |
|
1 2 3 4 |
Name Result URL Magento Commerce 404 http://www.magentocommerce.com/var/log/system.log Cafe 301 http://www.cafe.com/var/log/system.log Lunch 503 http://www.lunch.com/var/log/system.log |
It’s a nightmare. Your production environment was compromised and you actually don’t know how and how much data was stolen. For sure there are different ways to be compromised. In this post I just want to explain, how hackers can get full access to your production environment by using image files.
As explained on Snapfast and Sucuri it is pretty easy to store any kind of PHP code in EXIF headers. Often times it only needs a simple script to create an administrator account. If you are using third-party extensions with an image upload function or if you are late with the last security update, there is a high risk to be compromised.
You can use commands like svn status or git status to find changes, but this is no guarantee to find malicious code, because oftentimes your /media/ or similar folders are not version controlled.
Here is one simple command how you can find infected image files.
|
1 2 |
$ cd /var/www/{{{MAGENTO}}}/ $ find . -iregex '.*\.\(jpg\|gif\)' -exec grep -o -P 'base64_decode' {} \; |
In case if any file is infected, you will see the following search result.
|
1 |
Binary file ./media/test.jpg matches |
Create a manually backup of media/catalog without /cache/ folder
|
1 2 |
$ cd /var/www/{{{MAGENTO}}}/media/ $ tar -czf ~/catalog.tar.gz catalog --exclude 'catalog/product/cache/*' |
The same with the current date in the filename.
|
1 |
$ tar -czf ~/"catalog_$(date '+%Y%m%d').tar.gz" catalog --exclude 'catalog/product/cache/*' |
Find files which are not owned by apache group or user. This is useful if you want detect possible permission issues.
|
1 2 |
$ cd /var/www/{{{MAGENTO}}}/ $ find . \! -group www-data \! -user www-data |
You should never left database dump or other export files with sensible information in your document root, because there is a potential risk that those files can be downloaded from outside, depending on your security settings. To make sure your production environment is save, just run a simple ” find ” with the option ” iregex “.
|
1 2 |
$ cd /var/www/{{{MAGENTO}}}/ $ find -iregex '.*\.\(sql\|tar\|gz\|zip\|csv\)' |
|
1 2 |
$ cd /var/www/{{{MAGENTO}}}/var/report/ $ find . -type f -ctime -1 |
Here are some more options you can use for ” -ctime “.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
# find files created greater than 24 hours ago find . -type f -ctime +0 # find files created between now and 1 day ago # (i.e., in the past 24 hours only) find . -type f -ctime 0 # find files created less than 1 day ago find . -type f -ctime -1 # find files created between 24 and 48 hours ago find . -type f -ctime 1 # find files created more than 48 hours ago find . -type f -ctime +1 |
A fast growing system.log or exception.log can be the cause of a server outage. If you want empty log files on a production environment, the command ” truncate ” will help you.
|
1 2 |
$ cd /var/www/{{{MAGENTO}}}/var/log $ truncate -s 0 system.log |
For further debugging you can keep log data by specifying the size.
|
1 |
$ truncate -s 10M system.log |
Improve your bug fixing routine with some simple grep commands. Run these commands before any production deployment just to make sure your new code is working properly. To reduce the output on your terminal, extend my examples as you can see below.
Count the results with ” wc -l ” at the end.
|
1 |
$ grep -oi 'undefined index' system.log | wc -l |
Display the 1000 rows with ” tail -n 1000 ” at the beginning.
|
1 |
$ tail -n 1000 system.log | grep -i 'undefined index' |
1.1 Access to an array by a key that doesn’t exist
|
1 |
$ grep -i 'undefined index' system.log |
1.2 Use a variable that wasn’t previously defined
|
1 |
$ grep -i 'undefined variable' system.log |
1.3 Debug output generated by Mage::log()
|
1 |
$ grep -i 'debug' system.log |
1.4 Any kind of permission issues
|
1 |
$ grep -i 'permission denied' system.log |
1.5 Missing Magento template files
|
1 |
$ grep -i 'not valid template file' system.log |
1.6 Access to a non-object
|
1 |
$ grep -i 'trying to get property of non-object' system.log |
1.7 Missing argument in a function call
|
1 |
$ grep -i 'missing argument' system.log |
1.8 Invalid argument supplied to a loop
|
1 |
$ grep -i 'invalid argument supplied' system.log |
Find all:
|
1 |
$ cat system.log | egrep -i '(debug|notice: undefined|permission denied|not valid template file|trying to get property of non-object|missing argument|invalid argument supplied)' |
2.1 Any kind if XML issues
|
1 |
$ grep -i 'xml schema' exception.log |
2.2 Specific Authorize.net gateway issues
|
1 |
$ grep -i 'paygate.*gateway' exception.log |
2.3 Specific PayPal gateway issues
|
1 |
$ grep -i 'paypal.*gateway' exception.log |
2.4 Find general curl issues
|
1 |
$ grep -i 'curl' exception.log |
Find all:
|
1 |
$ cat exception.log | egrep -i '(xml schema|paygate.*gateway|paypal.*gateway|curl)' |
Find the last 10 error report files and grep for any kind of error message.
3.1 MySQL connection issues
|
1 |
$ ls -1t | head -10 | xargs grep -i 'mysql server' -sl |
3.2 Invalid config field
|
1 |
$ ls -1t | head -10 | xargs grep -i 'invalid config field' -sl |
3.3 Unable to read response
|
1 |
$ ls -1t | head -10 | xargs grep -i 'unable to read response' -sl |
3.4 External urls redirect
|
1 |
$ ls -1t | head -10 | xargs grep -i 'external urls redirect' -sl |
Find all:
|
1 |
ls -1t | head -10 | xargs egrep -i '(mysql server|invalid config field|unable to read response|external urls redirect)' -sl |
4.1 Find missing files caused by any reason
|
1 |
$ tail -n 1000 /var/log/httpd/{virtualhost}-error_log | grep -i 'file does not exist' |
As long as you don’t need root permissions in your PHP application / script, it is recommended to set up cron jobs with a web server specific user.
Get the current apache user from the process status.
|
1 2 3 |
$ ps axo user,group,comm | egrep '(apache|httpd)' root root apache2 www-data www-data apache2 |
Edit the crontab with the option -u and the specific apache user. This command will create a user-specific cron file in /var/spool/cron/crontabs/.
|
1 |
$ crontab -u www-data -e |
Set up your crontab and save the changes with CTRL-O.
|
1 2 3 4 5 6 7 8 |
# For example, you can run a backup of all your user accounts # at 5 a.m every week with: # 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/ # # For more information see the manual pages of crontab(5) and cron(8) # # m h dom mon dow command */5 * * * * /bin/sh /var/www/magento/cron.sh |
After that, check the syslog file to make sure that the cronjob is running correctly.
|
1 |
$ grep 'CRON'.*\(www-data\) /var/log/syslog |
If your hosting environment has no database monitoring, you should be able to monitor it manually once in a while or with your own shell script to prevent rapidly growing MySQL tables, because the result of large MySQL tables ( I am talking about 2 – 10 GB with millions of rows ) are often random performance issues, especially if you are using community modules with not well written MySQL queries.
|
1 2 3 4 |
SELECT TABLE_NAME, table_rows, data_length, index_length, round(((data_length + index_length) / 1024 / 1024),2) "Size in MB" FROM information_schema.TABLES WHERE table_schema = "{{{YOUR_DATABASE}}}" ORDER BY (data_length + index_length) DESC; |
This is the output of the query.
|
1 2 3 4 5 6 7 8 9 10 |
| enterprise_logging_event | 122389 | 15939572 | 3270656 | 18.32 | | catalog_product_index_eav | 324014 | 4212182 | 18053120 | 21.23 | | log_visitor_info | 182838 | 19706548 | 3599360 | 22.23 | | catalog_product_index_eav_idx | 324014 | 4522245 | 21139456 | 24.47 | | catalog_product_entity_varchar | 255303 | 10852180 | 15244288 | 24.89 | | log_url | 228886 | 10966704 | 15636480 | 25.37 | | log_url_info | 228886 | 80258352 | 7542784 | 83.73 | | enterprise_logging_event_changes | 69184 | 160619752 | 1499136 | 154.61 | +-----------------------------------------------+------------+-------------+--------------+------------+ 586 rows in set (0.05 sec) |
Don’t freak out if your Magento is not working and because of the following error message.
|
1 |
SQLSTATE[HY000]: General error: 145 Table './magento/log_visitor_info' is marked as crashed and should be repaired |
The reason of a crashed MySQL table is mostly related to a server issue. For example:
In most cases you can repair the affected table with a simple repair command, which comes with MySQL.
|
1 2 3 4 5 6 7 |
mysql> repair table log_visitor_info; +-----------------------+--------+----------+----------+ | Table | Op | Msg_type | Msg_text | +-----------------------+--------+----------+----------+ | magento.log_visitor_info | repair | status | OK | +-----------------------+--------+----------+----------+ 1 row in set (0.54 sec) |
This command looks pretty easy and safe, but keep in mind that you should always create a backup before you change anything in your database, because probably more MySQL tables are affected.