понедельник, 9 ноября 2020 г.

PHP-FPM via command line

1. Install apt install libfcgi0ldbl

2. Install apt install html2text

3. Sh script

SCRIPT_FILENAME=/path/to/site/php_info.php \
REQUEST_URI=/ \
QUERY_STRING= \
REQUEST_METHOD=GET \
cgi-fcgi -bind -connect /run/php/php7.3-fpm.sock

4. script.sh | html2text

We will get phpinfo() function output through php-fpm, not through php-cli

четверг, 25 июня 2020 г.

PHP Xdebug multi php versions in the server , fixing xdebug.so: undefined symbol: zend_post_startup_cb

If you have many php versions in the server we might face Xdebug installing.
Firstly, I installed via manual as
install PECL, then install xdebug through pecl
As result, I got an error

xdebug.so: undefined symbol: zend_post_startup_cb

The default php version in the server is PHP 7.3 . I need to install Xdebug for PHP 7.2
I used those links
https://gist.github.com/amenk/29636622c60a420330a8b827d166f9cf
https://qna.habr.com/q/729423
https://xdebug.org/docs/install
The server is Debian. I installed php7.2-dev. We will use phpize and php-config from the package.
It is MATTER to use version as  phpize7.2 and php-config7.2
1) download
2) tar -xzf xdebug-2.9.6.tgz && cd xdebug-2.9.6
3) phpize7.2
4) ./configure --with-php-config=/usr/bin/php-config7.2
5) make
6) copy from modules folder to lib folder.
to check that lib folder via phpize7.2
7) add to available modules with 20 priority
PS: firstly, I used 10 priority and got error with IonCube. 20 priority helped
Then restart PHP service and enjoy

среда, 20 мая 2020 г.

Nginx map $request_uri example

If I need to add http auth to special location I can use map

http block

map $request_uri $auth {
    /2.3.5/  "Authorization Required";
    default  "off";
}

server block

    auth_basic $auth;
    auth_basic_user_file /etc/nginx/.htpasswd;

среда, 29 апреля 2020 г.

Magento 500 error Premature end of script headers: index.php, Error parsing script headers, (22)Invalid argument

[Wed Apr 29 13:55:43.181652 2020] [proxy_fcgi:error] [pid 113977] [client 127.0.0.1:38830] Premature end of script headers: index.php, referer: https://domain.tld/us/catalogsearch/result/?q=jacket

[Wed Apr 29 13:55:43.181721 2020] [proxy_fcgi:error] [pid 113977] [client 127.0.0.1:38830] AH01070: Error parsing script headers, referer: https://domain.tld/us/catalogsearch/result/?q=jacket

[Wed Apr 29 13:55:43.181726 2020] [proxy_fcgi:error] [pid 113977] (22)Invalid argument: [client 127.0.0.1:38830] AH01075: Error dispatching request to : , referer: https://domain.tld/us/catalogsearch/result/?q=jacket

Issue https://github.com/magento/magento2/issues/7657
https://cloud.githubusercontent.com/assets/3441674/20826784/5002f712-b8a8-11e6-95cf-ab8d9d94e239.png

header X-Magento-Tags size more than header size limit

среда, 22 апреля 2020 г.

Magento rewrite 404 error with checkout

It was dev site with enabled basic auth
Access log displayed 404 error
There was an error in nginx error log
 is not found (2: No such file or directory)

Working Apache config was
<IfModule mod_rewrite.c>
    Options +FollowSymLinks
    RewriteEngine on
    RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
    RewriteCond %{REQUEST_METHOD} ^TRAC[EK]
    RewriteRule .* - [L,R=405]
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-l
    RewriteRule .* index.php [L]
</IfModule>

I was moving to Nginx and got an error 404, because I only disabled auth with
location /onestepcheckout/ {
    auth_basic off; 
}
I had to add auth off for checkout, but I forgot about rewrite
try_files $uri/ $uri/ /index.php$is_args$args;
It was a reason.
Resolving error was

location / {     
    try_files $uri/ $uri/ /index.php$is_args$args;
}

location /onestepcheckout/ {
    auth_basic off;
    try_files $uri $uri/ /index.php$is_args$args;
}

# POST response
location /rest/ {
    auth_basic off;
    try_files $uri $uri/ /index.php$is_args$args;
}

пятница, 17 апреля 2020 г.

Magento CORS error (Access-Control-Allow-Origin)

https://absolutecommerce.co.uk/blog/cors-in-magento-2

/etc/nginx/magento2-cors.conf
add_header 'Access-Control-Allow-Origin' '*' 'always'; 
if ($request_method = 'OPTIONS') { 
  add_header 'Access-Control-Allow-Origin' '*' 'always'; 
  add_header 'Access-Control-Allow-Headers' 'x-requested-with' 'always'; 
  add_header 'Access-Control-Max-Age' 86400 'always'; 
  add_header 'Content-Length' 0 'always'; return 204; 
}

location /static/ { 
  location ~* \.(ico|jpg|jpeg|png|gif|svg|js|css|swf|eot|ttf|otf|woff|woff2)$ { 
    add_header X-Frame-Options "SAMEORIGIN"; 
    include /etc/nginx/magento2-cors.conf; 
  } 
  location ~* \.(zip|gz|gzip|bz2|csv|xml)$ { 
    add_header X-Frame-Options "SAMEORIGIN"; 
    include /etc/nginx/magento2-cors.conf; 
  } 
  add_header X-Frame-Options "SAMEORIGIN"; 
  include /etc/nginx/magento2-cors.conf; 
}

explanation CORS:

Header Access-Control-Allow-* - это заголовки ответа, их должен отправлять сервер в ответ на запрос. 
Если запрос OPTIONS (браузер шлет сам перед основным запросом для проверки прав), то сервер должен так же ответить этими заголовками, но со статусом 204 и без тела
с клиента должен быть только заголовок Origin (откуда пришел запрос), если клиент в браузере, то браузер сам его отправит (X-Requested-With кстати тоже сам отправит)
а эти:
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods" : "GET,POST,PUT,DELETE,OPTIONS",
"Access-Control-Allow-Headers": "Content-Type, Access-Control-Allow-Headers, Authorization, X-Requested-With"
должны быть с сервера, притом из Access-Control-Allow-Headers можно смело убрать Access-Control-Allow-Headers и X-Requested-With (первый Вы и не должны отправлять, а второй разрешен по умолчанию)

четверг, 9 апреля 2020 г.

Magento 2 Some links from staging home page redirect to the production site

Some links from staging home page redirect to the production site.
Analyzing...
It is home page, so, I have to check cms* tables.
I checked with

select title,content from cms_block where content like '%www.prod.com%';

and found three records
I modified them with  variable {{config path="web/secure/base_url"}} instead of prod domain.
It could be done via Admin Panel or via

Cleared FPC cache
UPDATE cms_block set content = REPLACE(content, 'https://www.prod.com/', '{{config path="web/secure/base_url"}}') WHERE content LIKE '%https://www.prod.com/%';

вторник, 31 марта 2020 г.

Siege Load-testing

Siege command line:

siege -f links.csv -c 150 -t15m -d 7 -iv -m "Company1 Mar 31 2020" -l

-c - concurrent users
-t15m - test time
-d 7 - 7sec delay for each user
-i - INTERNET user simulation, hits URLs randomly
-m - MARK, mark the log file with a string
-l - log file, if it isn't specified, then default is used: /var/siege.log

I'm increasing concurrent users number with step 50 within each test.

Tips&tricks:

I got unexpected good results with higher concurrent users number after previous bad with lower ones.
The reason were FPC cache (redis in my case) and OpCache.
So, tricks are cleaning FPC and OpCache caches before each test.
In my case,
redis-cli -p 6380 FLUSHALL 
and restarting php

понедельник, 23 марта 2020 г.

Jmeter plugins

Useful links
https://thetesttherapist.com/2018/07/14/get-max-number-of-user-concurrency-in-jmeter/
https://www.logicify.com/en/blog/apache-jmeter-a-powerful-tool-for-performance-and-load-testing/

Install
https://jmeter-plugins.org/install/Install/
Configure
https://jmeter-plugins.org/wiki/PluginInstall/#Configure-JMeter
Install Active Threads Over Time Listener
https://jmeter-plugins.org/wiki/ActiveThreadsOverTime/
Pick "3 Basic Graphs" in Plugins Manager

Additionally, considering 404 as an acceptable response
https://i.stack.imgur.com/hCLl3.png
https://stackoverflow.com/questions/31317077/how-to-test-an-expected-404-response-with-jmeter


Magento Site Load-Testing via Jmeter

1) gather links of the site;

All needed links are in url_rewrite table. I need the result file with random link lines from the table.
So, I will use RAND() function in select query. It will be enough if I use only category, product, cms_page links, so I will use such entity_type from the table as ones. I need avoid duplicates, so I will use distinct in the query.

    Finally, the result query is
mysql -uuser_name -p -hdatabase -e "select distinct request_path from url_rewrite where entity_type='caterory' or entity_type='product' or entity_type='cms_page' order by RAND();" db_name > load_test.log

2) I noticed that product records begins with slash as the fist character. I need to remove it.
 Such script should help there

./remove_first_slash.sh > load_test-ready.log

#!/bin/bash
for request_path in $(cat load_test.log); do
    tmp_val=$(echo "$request_path" | cut -c -1)
    if [[ "$tmp_val" == "/" ]]; then
        tmp_clear=$(echo "$request_path" | cut -c 2-)
        echo "$tmp_clear"
        continue
    fi
    echo "$request_path"
done

3) use it with Jmeter

Tips&tricks:

I got unexpected good results with higher concurrent users number after previous bad with lower ones.
The reason were FPC cache (redis in my case) and OpCache.
So, tricks are cleaning FPC and OpCache caches before each test.
In my case,
redis-cli -p 6380 FLUSHALL 
and restarting php

пятница, 20 марта 2020 г.

Apache Jmeter

Useful links:
https://jmeter.apache.org/usermanual/build-web-test-plan.html
https://www.blazemeter.com/blog/three-ways-to-generate-random-variables-in-jmeter/
https://www.blazemeter.com/blog/how-run-load-test-50k-concurrent-users/
https://www.blazemeter.com/blog/what%e2%80%99s-the-max-number-of-users-you-can-test-on-jmeter/
https://www.xtivia.com/blog/fixing-jmeter-socket-errors/

Http Cookie Manager

clear cookie

Http Header Manager

User-agent
Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5355d Safari/8536.25

CSV Data Set Config

Visible Names: request1

Http request

Path: ${request1}

Results

Vew Result Tree
View Results in Table
Summary Report
Aggregate Report

понедельник, 3 февраля 2020 г.

PHP Ioncube extension

It uses for encoding a code
1) Check an extension folder
php -i | grep extension_dir
2) download lib from https://www.ioncube.com/loaders.php site
3) make ini file in mods-available folder with
zend_extension = /usr/lib/php/20170718/ioncube_loader_lin_7.2.so
4) make symlinks to apache2 cli fpm folders
5) restart php, nginx, apache2

вторник, 21 января 2020 г.

Composer dump-autoload

Magento 2.3.3 Commerce
Useful links
https://phpprofi.ru/blogs/post/52
https://getcomposer.org/doc/03-cli.md#dump-autoload-dumpautoload-
https://magento.stackexchange.com/questions/284911/cant-run-any-commands-in-terminal/301796#301796

After

composer dump-autoload --optimize
php7.2 bin/magento setup:di:compile

throws error
Class Magento\Framework\App\ResourceConnection\Proxy does not exist

It works with composer dump-autoload
Also, I noticed if I run composer dump-autoload --optimize twice , second running was longer than first and compiling worked afterwards.

пятница, 10 января 2020 г.

Nginx security headers

Usually, I added

X-Frame-Options SAMEORIGIN

There was a task with CWE-693: Protection Mechanism Failure
I added security headers for strong security.

Useful links
https://www.keycdn.com/blog/http-security-headers
https://devdocs.magento.com/guides/v2.3/config-guide/secy/secy-xframe.html
magento 1 since 1.9.2 version

Tested the site headers via https://securityheaders.com/
Added to nginx.conf in
http block

add_header x-xss-protection "1; mode=block" always;

to vhost.conf in
server block

add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header x-content-type-options "nosniff" always;
add_header feature-policy "autoplay 'none'; camera 'none'" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;

Links related Referrer-Policy
https://scotthelme.co.uk/a-new-security-header-referrer-policy/
http://okiseleva.blogspot.com/2018/07/referrer-policy.html

It's the most difficult security header Content-Security-Policy

add_header Content-Security-Policy "default-src 'self'; img-src * 'self' data:; style-src 'self' 'unsafe-inline'; script-src * 'unsafe-inline' 'unsafe-eval'; frame-src *; object-src *; connect-src *; media-src *; font-src *;";

Links related Content-Security-Policy
https://benjaminhorn.io/code/content-security-policy-what-it-is-what-it-does-and-how-to-implement-it/
https://ole.michelsen.dk/blog/secure-your-website-with-content-security-policy.html

Error with img-src https://forum.framework7.io/t/content-security-policy-directive-img-src-self-data-https/5678

Short instruction for setting Content-Security-Policy header parameters:

Set to default-src 'none';
Open develop panel F12, check console log and add rules with all supported directives

Supported directives

default-src: Define loading policy for all resources type in case of a resource type dedicated directive is not defined (fallback) script-src: Define which scripts the protected resource can execute object-src: Define from where the protected resource can load plugins style-src: Define which styles (CSS) the user applies to the protected resource img-src: Define from where the protected resource can load images media-src: Define from where the protected resource can load video and audio frame-src: Define from where the protected resource can embed frames font-src: Define from where the protected resource can load fonts connect-src: Define which URIs the protected resource can load using script interfaces