You can add one or more statuses in settings.py. Scrapy will process requests normally and when one of these statuses is encountered, it will retry that request. You can modify RETRY_HTTP_CODES and add any number of statuses there. You can also control how many times to try with RETRY_TIMES

Following method can be used for creating a new user in Apache. This comes in real handy for my projects that user Apache authentication. I run following command when I’m asked to create a new user. htpasswd /etc/httpd/.htpasswd username_here

Add this to your .bashrc (Linux) or .bash_profile (MacOS): export HISTFILESIZE=-1 export HISTSIZE=-1

First install zip if not already install Ubuntu apt-get install zip Centos yum install zip Then create a file.sh and paste following code OUTFILE=outputfile.zip INPUTDIR=inputdir zip -R $OUTFILE $INPUTDIR scp $OUTFILE usrename@outserver:/path/of/ $OUTFILE -i key.pem

I love working with Linux and CLI. It helps boost speed to a great extent and it feels like Windows’ xampp often limits that because I mostly tend to avoid using command line. I’m finally writing this to change that and be equally as proficient on Windows. MySQL Bin Location {$xampp}/mysql/bin/mysql.exe Mysqldump Bin Location {$xampp}/mysql/bin/mysqldump.exe How to import and export files Import files Login to mysql by running {$xampp}/mysql/bin/mysql.exe -u user -p Select database by running use {$database_name} Import a file by running source {$path_to_file.sql} Export files Export a table by running mysql\bin\mysqldump.exe -u root -p database_name table > table.sql

A list of common terminal shortcuts + most common cli programs in Linux terminal ctrl + a move to start of a line ctrl + e move to end of a line Alt-f Move the cursor forward by one word Alt-b Move the cursor backward by one word ctrl + u Delete an entire line ctrl + k Delete everything after cursor Ctrl + Shift + F Find any text clear or ctrl + l to clear terminal reset clear everything and remove scroll cal show calendar command; command2 for running multiple commands regardless of output command && command2 for running commands only when first is successful ...

$query = "alter table {table} drop column columnName;"; $tables = array('tableA', 'tableB'); foreach ($tables as $key => $source) { $currentQuery = str_replace('{table}', $source, $query); DB::query($currentQuery); }

sudo apt update sudo apt install apt-transport-https ca-certificates curl software-properties-common curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable" sudo apt update sudo apt install docker-ce sudo systemctl status docker docker --version

Def vars const = cannot redeclare, cannot changed value let = can change value, cannot redeclare let + const = if inside { } then it won’t work outside Concatination no need for + sign use backticks instead of quotes you can use single or double quotes inside backticks you can use variables inside of backticks like My name is ${name} Object Literals If you want to return an object you don’t need to do following ...

Let’s say $xampp = 'xampp_installation_directory' Open {$xampp}\apache\conf\extra\httpd-vhosts.conf Add following snippet for a normal project <VirtualHost *:80> DocumentRoot "{$xampp}\htdocs\project_name" ServerName project_name.localhost <Directory "F:\xampp\htdocs\project_name"> Options Indexes FollowSymLinks AllowOverride All Require all granted </Directory> </VirtualHost> Add following snippet for a Laravel project <VirtualHost *:80> DocumentRoot "{$xampp}/htdocs/project_name/public" ServerName project_name.localhost <Directory "F:\xampp\htdocs\lara8/project_name"> Options Indexes FollowSymLinks AllowOverride All Require all granted </Directory> </VirtualHost>

You can execute same code with a conditional if you want multiple endpoints hiting the same functions. This is awesome because it allows you to reduce code duplication and the same block of code can do multiple things with minor changes. Let’s say you want to diplay a page on following two urls site.com/blog/page and site.com/page @app.route('/blog/<slug>', endpoint='post') @app.route('/<slug>', endpoint='page') def post(slug): post = getPost(slug) if request.endpoint == 'post': title = post['title'] elif request.endpoint == 'page': title = 'This is a page' return render_template('post.html', post=post, title=title)

You can use url_for function for building URLs. For example, let’s say you have a assets folder called static in main app directory. You can use following code in template files to include CSS files {{ url_for('static', filename='css/style.css') }}

if (!$argv['1']) { exit("php rotate_360.php input_dir output_dir degress_to_rot"); } print_r($argv); $input_dir = $argv['1']; $output_dir = $argv['2']; if (!file_exists($output_dir)) { @mkdir($output_dir); } $degress = !empty($argv['3']) ? $argv['3'] : 360; $files = glob("{$input_dir}/*.mp4"); foreach ($files as $key => $file) { $filename = pathinfo($file, PATHINFO_FILENAME); // outputs nameonly $command = "ffmpeg -i {$input_dir}/{$filename}.mp4 -c copy -metadata:s:v:0 rotate={$degress} {$output_dir}/{$filename}-out.mp4"; echo $command . "\n"; shell_exec($command); }

require 'vendor/autoload.php'; use Mailgun\Mailgun; function sendMail($params) { $from = isset($params['from']) ? $params['from'] : 'default@from.com'; $mailParams = array(); $attachments = array(); $mailParams['from'] = $params['sender_name'] . " <$from>"; $mailParams['to'] = $params['to']; #$m->setFrom($params['sender_name' ] . " <$from>"); if (isset($params['attachment'])) { $attachments[] = array('filePath' => $params['attachment']['path'], 'filename' => $params['attachment']['name']); } if (isset($params['custom_attachment'])) { foreach ($params['custom_attachment'] as $key => $file) { $attachments[] = array('filePath' => $file['path'], 'filename' => $file['name']); } } if (isset($params['replyTo'])) { if (is_array($params['replyTo'])) { foreach ($params['replyTo'] as $key => $email) { $mailParams['h:Reply-To'][] = $email; } } else { $mailParams['h:Reply-To'] = $params['replyTo']; } } $mailParams['subject'] = $params['subject']; $isHtml = $params['message'] != strip_tags($params['message']); if ($isHtml) { $mailParams['html'] = $params['message']; } else { $mailParams['text'] = $params['message']; } if (!empty($attachments)) { $mailParams['attachment'] = $attachments; } $mg = Mailgun::create('api_key_here'); $status = $mg->messages()->send('mail.collectiveproperties.net', $mailParams); return $status; }// sending email $params = array(); $params['to'] = 'frank@court.com'; $params['from'] = 'wilson@prison.com'; $params['sender_name'] = 'Frank Castle'; $params['subject'] = 'IM COMING FOR YA'; $params['message'] = 'One batch! Two batch! Penny & dime!'; $params['replyTo'] = 'matt@nelsonandmurdock.com'; $params['attachment'] = array('path' => 'test.jpg', 'name' => 'test.jpg'); sendMail($params);

Center text Structure your HTML like this <div class="container"> <p>Hello, (centered) World!</p> </div> And then use following CSS p { text-align: center; } Center a Div with CSS Margin Auto <div class="container"> <div class="child"></div> </div> .child { margin: 0 auto; } Center a Div Horizontally with Flexbox <div class="container"> <div class="child"></div> </div> .container { display: flex; justify-content: center; }

add subtitles You can hardcode subtitles by providing a subtitles files and running the following command ffmpeg -i input.mp4 -vf subtitles=subs.srt out.mp4 add watermark in any position The base command for adding watermark looks like this ffmpeg -i input.mp4 -i watermark.png -filter_complex "POSITION_HERE" out.mp4 You can replace POSITION_HERE with any position you need from the following Top Right overlay=main_w-overlay_w-10:10 Top Left overlay=10:10 Bottom Right overlay=main_w-overlay_w-10:main_h-overlay_h-10 Bottom Left overlay=10:main_h-overlay_h-10 Center overlay=x=(main_w-overlay_w)/2:y=(main_h-overlay_h)/2 extract audio If you need to extract audio of a video with re-encoding the whole thing then you can run following command ...

Two create side by side items, wrap items in a .row div <div class="row"> <div> <!-- element 1 here --> </div> <div> <!-- element 2 here --> </div> </div>

If you are dealing with requesting images or data from other websites and you are getting blocked, you must start using proxies. Here are the fields that you can start sending with CURL curl_setopt($ch, CURLOPT_PROXY, $proxy); curl_setopt($ch, CURLOPT_PROXYPORT, $port); // if your proxy service requires authentication $autentication = 'user:pass'; curl_setopt($ch, CURLOPT_PROXYUSERPWD, $autentication); curl_setopt($ch, CURLOPT_PROXYTYPE, 'HTTP'); curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);

<table class="table table-bordered"> <thead> <tr> {% for range, dets in rents %} <th scope="col">{{range}}</th> {% endfor %} </tr> </thead> <tbody> <tr> {% for range, details in rents %} <td scope="row">{{details.item}}</td> {% endfor %} </tr> </tbody> </table>

find directory -type f -printf '%TY-%Tm-%Td\n' | sort | uniq -c

wget --password=pass_here --user=user_name_here file_to_download

for i in {1..number_of_loops}; do commandsSeparatedBySemicolon; done;

This article offers a sample of basic Markdown syntax that can be used in Hugo content files, also it shows whether basic HTML elements are decorated with CSS in a Hugo theme. ...

<?php $file = 'data.csv'; $separator = "\t"; $done = 0; if (($handle = fopen($file, "r")) !== FALSE) { while (($data = fgetcsv($handle, false, "$separator")) !== FALSE) { $done += 1; if ($done < 2) { $fields = explode($separator, str_replace(' ', '', implode($separator, $data))); continue; } $item = array_combine($fields, $data); print_r($item); } }

Following method can be used to center any element. It comes really handy when building pagination buttons that are generally place at the bottom of page right in the middle. <div class="text-center"> <button class="btn btn-primary"></button> </div>

Following method can be used to retrieve password of a connection that you can automatically connect to because Windows remembers the password but you don’t. You need to share password with a friend or you’d like to connect from other device. Just open powershell and run following command. netsh wlan show profile name=profilename key=clear

Sphinx search is a tool that extends Mysql’s FULLTEXT search capabilities and brings faster more meaningful results in your control. Start /usr/bin/searchd Stop /usr/bin/searchd --stop Find status /usr/bin/searchd --status Re-index all data sudo indexer --all

function randomString($length = 10) { $characters = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'; $charactersLength = strlen($characters); $randomString = ''; for ($i = 0; $i < $length; $i++) { $randomString .= $characters[rand(0, $charactersLength - 1)]; } return $randomString; }

Go to chrome://extensions/ Find extension and copy ID Go to %localappdata% C:\\Users\\{username}\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Extensions Find extension folder Bingo!