The New Method of Squid to Cache Youtube

This is the last method of Squid that I know to cache youtube. Thanks for someone, his name is.... (sorry I forget his name.) He gave me a link website. And he asked me to try to cache youtube, according the method on the website.
Ok... I have tried it... and I did it... successfully. Hahahaha...

Perbandingan Performance Squid 3 dan Squid 2

Masih seputar squid. Beberapa waktu yang lalu, saya terpaksa ngoprek linux lagi,.. setelah berbulan-bulan istirahat… Ini dikarenakan ada temen yang minta dibuatkan server proxy squid untuk di warnetnya… Tentu saja dia akan bayar… Saya berikan 1 syarat lagi, komputer untuk server proxynya dia beli juga dari saya (maksa, hahaha..) Akhirnya deal juga, lumayanlah tambahan duit :)

Idealnya, dengan adanya proxy squid di dalam network,

• Situs http dan https, bisa dicache.
• Youtube dan video lainnya bisa dicache.
• Dan yang paling penting adalah user bisa browsing dengan cepat dan persentase hit squid bagus.

Caching Youtube with storeurl.pl Method (100% Working in 2014)

Happy new year. Hopefully in 2014, we are all be healthy and success, always.

This is my first posting in 2014. Now I will discuss about caching google youtube video with storeurl.pl. Until now, caching youtube video with storeurl method, still 100% working, although not perfectly. Previously, I actually gave up, because 2 weeks before, caching youtube does not work with all methods (including methods storeurl). When the client runs youtube (through a server proxy), youtube error, as shown below. This possibility is from youtube site, there may be changes in the algorithm. Fortunately now no error anymore. Besides youtube, some other video sites can also be cached with this method. For example, dailymotion and so on.

Transparent Proxy with Squid 3.3.11 + Ubuntu + Shorewall + Mikrotik (Update)

This is my update posting, about Transparent Proxy with Squid 3.3.x + ubuntu + Shorewall + Mikrotik. Someone told me that the configuration worked but they had TCP_MISS, I was curious. Then I try to re-configure the above experiment with some recent software updates, and this is it.

Network configuration as shown above. I use virtual box to test it. Ubuntu server 13.10, mikrotik 5.20, shorewall 4.5.21.4, and squid 3.3.11. The instruction how to configure the network above, still same like at previous posting. 

How to make Squid 3.3.5 as Transparent Proxy Https / Ssl in Centos 6.4

The experiment continues to make transparent proxy https / ssl. Same as previous post. Look at "How to make Squid 3.1.23 as Tproxy (Transparent Proxy) in Centos 6.4". Do the step 1 and 2, ie install shorewall and install dhcp server. Then continue with the next steps.

How to make Squid 3.1.23 as Tproxy (Transparent Proxy) in Centos 6.4

From the title, we will only make squid tproxy (transparent proxy), but there are actually 3 work to be done.

1. Configuration of the firewall / shorewall so Linux Centos  will become a router.

2. Make dhcp server, and

3. Install squid as tproxy ( I use squid version 3.1.23 ).

We will work on gradually, so you can easily fix it if there is an error.

Cache Youtube Video with Squid 3.3.2 and Nginx

Note :

source : http://aacable.wordpress.com/2012/08/13/youtube-caching-with-squid-nginx/

I change some command, according with my network environment, and my experience.

( Update January 2th, 2014). Look at another posting, cache youtube video with storeurl method.  http://myconfigure.blogspot.com/2014/01/caching-youtube-with-storeurlpl-method.html

I assume you've installed :
1. Ubuntu 12.10,
2. Squid 3.3.2
3. Mikrotik, options to make it better
All of the above is installed and running nomally.If you have not installed some of the things mentioned above. Stop reading this section, and look for the section on this blog, about how to install squid. For example, you can read the
"Transparent Proxy with Squid 3.3.2 on Ubuntu 12.10 + Shorewall + Mikrotik".

Ok. Let's begin.

1. Add some command line at squid.conf

Add

Transparent Proxy with Squid 3.3.2 on Ubuntu 12.10 + Shorewall + Mikrotik

 

This post explains how to setup a transparent proxy (intercept proxy) by combining mikrotik and squid eksternal. Look at a network above. a mikrotik as router, coupled with a proxy server (squid as transparent proxy). In the previous article has been discussed about this network. You can read at "Ubuntu 12.10 + Squid (Transparent Proxy) + Shorewall + Mikrotik". But the squid that was used is version 3.1.23. If you replace it with squid version 3.2.8 or 3.3.2, would be an error. The reason is, the proxy misrepresented the "redirect".
So.  If you want to use squid 3.2.8 or 3.3.2 as a transparent proxy, in a network as above, then there are some things that should be changed, the configuration on the router mikrotik and the Squid.
A few steps below, would be similar to the steps at "Transparent Proxy Server, Squid 3.3.2 / 3.2.8 on Ubuntu 12.10 + Shorewall"
We begin by configuring the Ethernet.

vi /etc/network/interfaces

Transparent Proxy Server, Squid 3.3.2 / 3.2.8 on Ubuntu 12.10 + Shorewall

Squid 3.1.xx (latest version 3.1.23), is a very simple and is used to make transparent / intercept. While the squid 3.3.2 / 3.2.8, to make transparent / intercept configuration is a bit different from the version of squid 3.1.23.  In this blog, I will explain how to create a transparent / intercept in two ways, the first is squid as well as a router, and the second, Mikrotik as a router coupled with squid external. Let's get to start, a transparent proxy server with squid as well as router.

Network configuration.
After installing the new Ubuntu, configure the ethernet. In my case, Ethernet to the Internet is dhcp, and another ethernet to the LAN, using the IP address 10.10.10.1. Of course, customize your network environment.

vi /etc/network/interfaces

Squid 3.3.2 / 3.2.8 : Example Squid.conf

#WELCOME TO SQUID
#       ----------------------------

acl snmppublic snmp_community public

acl bamboe src 192.168.10.0/24

#uncomment noway url, if necessary.
#acl noway url_regex -i "/etc/squid/noway"

acl SSL_ports port 443

acl Safe_ports port 80          # http

acl Safe_ports port 1935          # http

acl Safe_ports port 21          # ftp

acl Safe_ports port 443         # https

acl Safe_ports port 70          # gopher

acl Safe_ports port 210         # wais

acl Safe_ports port 1025-65535  # unregistered ports

acl Safe_ports port 280         # http-mgmt

acl Safe_ports port 488         # gss-http

acl Safe_ports port 591         # filemaker

acl Safe_ports port 777         # multiling http

Clear Memory in Linux

This is a script to clear the memory. This script is typically useful for servers that use squid. Squid as a proxy so greedy with memory. Create a file to put this script, for example file "clearmemory". And place it in the folder /etc/cron.daily/, in order to run automatically every day.
vim /etc/cron.daily/clearmemory.

Monitoring Squid with Cacti (Template Cacti for Squid... Complete...!!!)

I assume that you have installed snmp, squid and cacti, before. I will not explain again about how to install squid, snmp, cacti, etc.. there's so many article to discuss about that’s.
Then, if all installation is complete, let's make graph template for squid.

1. cek your squid configuration,
# squid -v
Squid Cache: Version 3.1.22

Ubuntu 12.10 + Squid 3.1.23 (Transparent Proxy) + Shorewall + Mikrotik

Installation & Configuration

Step-by-step installation, almost the same as previous versions of ubuntu, can be checked from the video install ubuntu 12:04, below. Thank you for the uploader.

Squid 3.1.23 : Example Squid.conf & Squid start up

#Squid.conf

#WELCOME TO SQUID 3.1.23
#       ----------------------------
acl snmppublic snmp_community public
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32

acl bamboe src 192.168.3.0/24
#uncomment noway url, if necessary
#acl noway url_regex -i "/etc/squid/noway"
acl SSL_ports port 443
acl Safe_ports port 80          # http

Forwarding loop detected [Solve]

What is a forwarding loop?

A forwarding loop is when a request passes through one proxy more than once. You can get a forwarding loop if

a cache forwards requests to itself. This might happen with interception caching (or server acceleration) configurations.

a pair or group of caches forward requests to each other. This can happen when Squid uses ICP, Cache Digests, or the ICMP RTT database to select a next-hop cache.

Forwarding loops are detected by examining the Via request header. Each cache which "touches" a request must add its hostname to the Via header. If a cache notices its own hostname in this header for an incoming request, it knows there is a forwarding loop somewhere.

RRDtool 1.4.7 in ubuntu 10.04

Install RRDtool

Run the following commands as superuser:

sudo su

apt-get install libpango1.0-dev libxml2-dev

wget http://oss.oetiker.ch/rrdtool/pub/rrdtool-1.4.7.tar.gz
tar -zxvf rrdtool-1.4.7.tar.gz

cd rrdtool-1.4.7

mkdir /tmp/rrdbuild

export BUILD_DIR=/tmp/rrdbuild

mkdir /opt/rrdtool-1.4.7

export INSTALL_DIR=/opt/rrdtool-1.4.7

Traffic Shaping

Note : This is a good article about traffic shaping. Lets go to learn together… :) Source : http://blog.edseek.com/~jasonb/articles/traffic_shaping/scenarios.html

1. Introduction

1.1. Intended Audience

I have written this document with the experienced Linux administrator in mind. I make the assumption that the reader is familiar with compiling a kernel, downloading and configuring source packages from project Web sites, the Bash shell, patching sources, and so forth. It is also assumed that the reader has at least passing familiarity with Linux's Netfilter packet filter. Those already familiar with traffic control on other devices or platforms may wish to skip Introduction to QoS and start with Linux Traffic Control.

Missing File : startup squid

Sometimes, when you install a new version squid , you can not find "start-up" file.
That was happened to me. I used Ubuntu 10.04 LTS, and tried to install a new version squid, which I have previously downloaded from http://www.squid-cache.org. After installation was finished, there was no "start-up" file.
Below, start-up file, which I have modified. The original file is from squid Mandriva 2009.0. :)

#!/bin/bash