分类目录归档:Linux

upstream timed out (110: Connection timed out) while reading upstream

2018/04/26 23:34:39 [error] 10872#0: *268906 upstream timed out (110: Connection timed out) while reading upstream, client: 119.103.223.107, server: ****, request: “GET /**** HTTP/1.1”, upstream: “****”, host: “****”

#fastcgi_connect_timeout 600;
#fastcgi_read_timeout 600;
#fastcgi_send_timeout 600;
proxy_connect_timeout 600;
proxy_read_timeout 600;
proxy_send_timeout 600;

upstream timed out (110: Connection timed out) while reading upstream,

upstream timed out (110: Connection timed out) while reading response header from upstream

(24: Too many open files)

2018/04/26 23:32:58 [crit] 1457#0: *259351 open() “/usr/share/nginx/html/50x.html” failed (24: Too many open files), client: 112.17.247.117, server: ***, request: “GET /*** HTTP/1.1”, upstream: “***”, host: “***”

worker_rlimit_nofile 65535;

while read line无法读取最后一行的问题

while read line读取文件时,如果文件最后一行之后没有换行符\n,则read读取最后一行时遇到文件结束符EOF,循环终止,虽然此时$line内存有最后一行,但程序已经没有机会再处理此行,因此可以通过以下代码来解决此问题:

while read line || [[ -n ${line} ]]; do

done
这样当文件没有结束时不会测试-n $line,当遇到文件结束时,仍然可以通过测试$line是否有内容来进行继续处理。

http://ju.outofmemory.cn/entry/86710

MySQL自动备份到亚马逊S3


#!/bin/sh

# Updates etc at: https://github.com/woxxy/MySQL-backup-to-Amazon-S3
# Under a MIT license

# change these variables to what you need
MYSQLROOT=XXX
MYSQLPASS=XXXX
S3BUCKET=xx-mysql-backup
# DATABASE='--all-databases'
DATABASE=${1-default}
FILENAME=${DATABASE}
HOST=master
DUMPOPTION='--quick --single-transaction'
# the following line prefixes the backups with the defined directory. it must be blank or end with a /
S3PATH=mysql-backup/${DATABASE}
# when running via cron, the PATHs MIGHT be different. If you have a custom/manual MYSQL install, you should set this manually like MYSQLDUMPPATH=/usr/local/mysql/bin/
MYSQLDUMPPATH=
#tmp path.
TMP_PATH=/tmp/

DATESTAMP=$(date +".%Y-%m-%d.%H:%M:%S")
DELDATESTAMP=$(date -d"15 day ago 2017-04-16" +".%Y-%m-%d.%H:%M:%S")
DAY=$(date +"%d")
DAYOFWEEK=$(date +"%A")

PERIOD=${2-day}
if [ ${PERIOD} = "auto" ]; then
if [ ${DAY} = "01" ]; then
PERIOD=month
elif [ ${DAYOFWEEK} = "Sunday" ]; then
PERIOD=week
else
PERIOD=day
fi
fi

echo "Selected period: $PERIOD."

echo "Starting backing up the database to a .gz file..."

# dump all databases
${MYSQLDUMPPATH}mysqldump -h${HOST} ${DUMPOPTION} --user=${MYSQLROOT} --password=${MYSQLPASS} ${DATABASE} | gzip > ${TMP_PATH}${FILENAME}.gz

echo "Done backing up the database to a file."
# echo "Starting compression..."

# tar czf ${TMP_PATH}${FILENAME}${DATESTAMP}.tar.gz ${TMP_PATH}${FILENAME}.sql
mv ${TMP_PATH}${FILENAME}.gz ${TMP_PATH}${FILENAME}${DATESTAMP}.gz

# echo "Done compressing the backup file."

# upload all databases
echo "Uploading the new backup..."
s3cmd put -f --check-md5 -s --continue-put ${TMP_PATH}${FILENAME}${DATESTAMP}.gz s3://${S3BUCKET}/${S3PATH}${PERIOD}/
echo "New backup uploaded."
if [ $? -ne 0 ]
then
echo "Re uploading the backup file..."
s3cmd put -f --check-md5 -s --continue-put ${TMP_PATH}${FILENAME}${DATESTAMP}.gz s3://${S3BUCKET}/${S3PATH}${PERIOD}/
echo "Re upload backup file done."
fi

echo "Moving the backup from past $PERIOD to another folder..."
s3cmd mv --recursive s3://${S3BUCKET}/${S3PATH}${PERIOD}/${FILENAME}${DELDATESTAMP} s3://${S3BUCKET}/${S3PATH}previous_${PERIOD}/
echo "Past backup moved."

# we want at least two backups, two months, two weeks, and two days
echo "Removing old backup (15 ${PERIOD}s ago)..."
s3cmd del --recursive s3://${S3BUCKET}/${S3PATH}previous_${PERIOD}/
echo "Old backup removed."

# remove databases dump
# rm ${TMP_PATH}${FILENAME}.sql
echo "Removing the gz files..."
rm ${TMP_PATH}${FILENAME}${DATESTAMP}.gz
echo "Files removed."
echo "All done."

nginx: [emerg] unknown directive “proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for” in

$remote_addr 可信时才能信任 X-Forwarded-For、$proxy_add_x_forwarded_for

nginx配置代理转发,部分配置如下

        location / {
# index index.html index.htm index.php;
# try_files $uri $uri/ /index.php$request_uri;
proxy_set_header Host $host;
#proxy_set_header X-Real-IP $remote_addr;
#proxy_set_header X-Real-Port $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass https://10.29.185.61:8063;
}

$proxy_add_x_forwarded_forthe “X-Forwarded-For” client request header field with the $remote_addr variable appended to it, separated by a comma. If the “X-Forwarded-For” field is not present in the client request header, the $proxy_add_x_forwarded_for variable is equal to the $remote_addr variable.

“ X-Forward-For”客户端请求头字段,后面附加 $remote _ addr 变量,中间用逗号分隔。如果客户端请求头中没有“ X-Forward-For”字段,则 $proxy_add_x_forwarded_for 变量等于 $remote_addr 变量。

shadowsocks服务端快速搭建

Install (centos7)

yum install python-setuptools && easy_install pip
pip install git+https://github.com/shadowsocks/shadowsocks.git@master

Usage

ssserver -p 443 -k password -m aes-256-cfb

To run in the background:

sudo ssserver -p 443 -k password -m aes-256-cfb --user nobody -d start

To stop:

sudo ssserver -d stop

To check the log:

sudo less /var/log/shadowsocks.log

pecl update mongodb 1.1.* to 1.2.* 导致 php-fpm 全部立即段错误并失去控制,systemctl restart php-fpm失败

第二次遇到了~
ps -ef | grep php-fpm | cut -c 9-15 | xargs kill && systemctl restart php-fpm


Dec 15 20:40:02 web4 systemd: Removed slice User Slice of root.
Dec 15 20:40:02 web4 systemd: Stopping User Slice of root.
Dec 15 20:40:25 web4 kernel: traps: php-fpm[10171] general protection ip:7f4046d83120 sp:7ffc76043de0 error:0 in mongodb.so[7f4046d57000+be000]
Dec 15 20:40:25 web4 kernel: traps: php-fpm[1770] general protection ip:7f4054402542 sp:7ffc76045578 error:0 in ld-2.17.so[7f40543e9000+21000]
Dec 15 20:40:25 web4 kernel: traps: php-fpm[1771] general protection ip:7f4054402542 sp:7ffc76045578 error:0 in ld-2.17.so[7f40543e9000+21000]
Dec 15 20:40:25 web4 kernel: traps: php-fpm[1772] general protection ip:7f4054402542 sp:7ffc76045578 error:0 in ld-2.17.so[7f40543e9000+21000]
Dec 15 20:40:25 web4 kernel: traps: php-fpm[1773] general protection ip:7f4054402542 sp:7ffc76045578 error:0 in ld-2.17.so[7f40543e9000+21000]
Dec 15 20:40:25 web4 kernel: traps: php-fpm[1774] general protection ip:7f4054402542 sp:7ffc76045578 error:0 in ld-2.17.so[7f40543e9000+21000]
Dec 15 20:40:25 web4 kernel: traps: php-fpm[1775] general protection ip:7f4054402542 sp:7ffc76045578 error:0 in ld-2.17.so[7f40543e9000+21000]
Dec 15 20:40:25 web4 kernel: traps: php-fpm[1776] general protection ip:7f4054402542 sp:7ffc76045578 error:0 in ld-2.17.so[7f40543e9000+21000]
Dec 15 20:40:25 web4 kernel: traps: php-fpm[1777] general protection ip:7f4054402542 sp:7ffc76045578 error:0 in ld-2.17.so[7f40543e9000+21000]
Dec 15 20:40:25 web4 kernel: traps: php-fpm[1778] general protection ip:7f4054402542 sp:7ffc76045578 error:0 in ld-2.17.so[7f40543e9000+21000]
Dec 15 20:40:26 web4 systemd: Stopping LSB: starts php-fpm...

goaccess log-format 参数说明

log-format The log-format variable followed by a space or \t for tab-delimited, specifies the log format string.

SPECIFIERS

  • %x A date and time field matching the time-format and date-format variables. This is used when a timestamp is given instead of the date and time being in two separate variables.
  • %ttime field matching the time-format variable.
  • %ddate field matching the date-format variable.
  • %vThe server name according to the canonical name setting (Server Blocks or Virtual Host).
  • %eThis is the userid of the person requesting the document as determined by HTTP authentication.
  • %hhost (the client IP address, either IPv4 or IPv6)
  • %rThe request line from the client. This requires specific delimiters around the request (single quotes, double quotes, etc) to be parsable. Otherwise, use a combination of special format specifiers such as %m%U%q and %H to parse individual fields.
    • Note: Use either %r to get the full request OR %m%U%q and %H to form your request, do not use both.
  • %mThe request method.
  • %UThe URL path requested.
    • Note: If the query string is in %U, there is no need to use %q. However, if the URL path, does not include any query string, you may use %q and the query string will be appended to the request.
  • %qThe query string.
  • %HThe request protocol.
  • %sThe status code that the server sends back to the client.
  • %bThe size of the object returned to the client.
  • %RThe “Referer” HTTP request header.
  • %uThe user-agent HTTP request header.
  • %DThe time taken to serve the request, in microseconds.
  • %TThe time taken to serve the request, in seconds with milliseconds resolution.
  • %L The time taken to serve the request, in milliseconds as a decimal number.
  • %^Ignore this field.
  • %~Move forward through the log string until a non-space (!isspace) char is found.
  • ~hThe host (the client IP address, either IPv4 or IPv6) in a X-Forwarded-For (XFF) field.