Thursday, May 30, 2013

How to make Squid 3.3.5 as Transparent Proxy Https / Ssl in Centos 6.4

notification security https / ssl
The experiment continues to make transparent proxy https / ssl. Same as previous post. Look at "How to make Squid 3.1.23 as Tproxy (Transparent Proxy) in Centos 6.4". Do the step 1 and 2, ie install shorewall and install dhcp server. Then continue with the next steps.

Install openssl
yum install openssl openssl-devel
cd /etc/squid/
openssl req -new -newkey rsa:1024 -days 365 -nodes -x509 -keyout myconfigure.pem -out myconfigure.pem
Then you will be asked to enter information for your certificate request. Just enter, enter and enter to leave blank.
Country Name (2 letter code) [XX]:
State or Province Name (full name) []:
Locality Name (eg, city) [Default City]:
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:
Email Address []:
#

Install Squid
In this experiment, for making transparent proxy, I use squid version 3.3.5. Configure the squid 3.3.5, make sure option --enable-icap-client   --enable-ssl , is exist.
cd /usr/local/squid-3.3.5
./configure --prefix=/usr --exec-prefix=/usr --bindir=/usr/sbin --sbindir=/usr/sbin --sysconfdir=/etc/squid --datadir=/usr/share/squid --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/lib/squid --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --x-includes=/usr/include --x-libraries=/usr/lib --enable-shared=yes --enable-static=no --enable-carp --enable-storeio=aufs,ufs --enable-removal-policies=heap,lru --disable-icmp --disable-delay-pools --disable-esi --enable-icap-client --enable-useragent-log --enable-referer-log --disable-wccp --enable-wccpv2 --disable-kill-parent-hack --enable-snmp --enable-cachemgr-hostname=localhost --enable-arp-acl --disable-htcp --disable-forw-via-db --enable-follow-x-forwarded-for --enable-cache-digests --disable-poll --enable-epoll --enable-linux-netfilter --disable-ident-lookups --enable-default-hostsfile=/etc/hosts --with-default-user=squid --with-large-files --enable-mit=/usr --with-logdir=/var/log/squid --enable-http-violations --enable-zph-qos --with-filedescriptors=65536 --enable-gnuregex --enable-async-io=64 --with-aufs-threads=64 --with-pthreads --with-aio --enable-default-err-languages=English --enable-err-languages=English --disable-hostname-checks --enable-underscores --enable-ssl ; make; make install

useradd squid
touch /etc/squid/noway
cd /etc/squid/
mkdir swap
chown squid:squid /etc/squid/swap
mkdir /cache
chown squid:squid /cache
chown -Rf squid:squid /var/log/squid/
cd /etc/squid/
mv squid.conf squid.conf.old
vim squid.conf

For squid configuration, take it from http://myconfigure.blogspot.com/2013/03/squid-332-328-example-squidconf.html. Then look again at squid.conf, some variable must be change. Make sure some variable below is exist.
acl bamboe src 192.168.10.0/24
http_port 3128
http_port 3127 intercept
https_port 3129 intercept ssl-bump cert=/etc/squid/myconfigure.pem key=/etc/squid/myconfigure.pem
ssl_bump allow all
always_direct allow all
##update##updaate##update##
If you get error like below,
SECURITY NOTICE: auto-converting deprecated "ssl_bump allow <acl>" to "ssl_bump client-first <acl>" which is usually inferior to the newer server-first bumping mode. Update your ssl_bump rules. 
Fix it with changing
ssl_bump allow all
To
ssl_bump client-first all

vim /etc/init.d/squid
chmod 755 /etc/init.d/squid
chkconfig --add  squid
chkconfig  --level 235 squid on
squid -z
/etc/init.d/squid start

Reconfigure shorewall.
Remember, I assume, you have installed shorewall, as I asked from the beginning ( step 1, at "How to make Squid 3.1.23 as Tproxy (Transparent Proxy) in Centos 6.4" ). And then we just need to reconfigure the shorewall, like below.
rules
REDIRECT        loc     3127    tcp     www     -
REDIRECT        loc     3129    tcp     443     -

Because, my squid version is 3.3.x, and then I just want to make Transparent Proxy, not Tproxy, so, in shorewall configuration folder, remove the content of file, providers and tcrules. Make sure both file  is empty contents.  
Done. After restart, from user computer, try to browse any site https (email yahoo, email google, or facebook). For example, browse to https://mail.google.com, or https://www.facebook.com. If you can connect to the site https,  and then you will get notification about security, like picture above, that’s mean squid has successful configured with option ssl_bump as transparent proxy ssl. Traffic headed to port 443 / https, currently must go through squid proxy to get to the destination.You can view users who active access to internet through proxy, with command.
tail –f /var/log/squid/access.log
1369742866.442    710 192.168.10.200 TCP_MISS/200 503 GET https://mail.google.com/mail/channel/test? - HIER_DIRECT/173.194.38.182 text/plain
1369742866.468    772 192.168.10.200 TCP_MISS/200 3890 POST https://mail.google.com/mail/? - HIER_DIRECT/173.194.38.182 text/javascript
1369742867.154    619 192.168.10.200 TCP_MISS/200 1737 POST https://mail.google.com/mail/? - HIER_DIRECT/173.194.38.182 text/javascript
1369742867.838    652 192.168.10.200 TCP_MISS/200 1441 POST https://mail.google.com/mail/? - HIER_DIRECT/173.194.38.182 text/javascript
1369742868.695    820 192.168.10.200 TCP_MISS/200 6918 POST https://mail.google.com/mail/? - HIER_DIRECT/173.194.38.182 text/javascript

Noted.
For Squid versions 3.1 and 3.2, both of them can be used, should be no problem. I was just starting to experiment with making transparent https / ssl. Probably much less configuration and less security. Please info, if there is a better option.

Reference
http://wiki.squid-cache.org/Features/DynamicSslCert. I edit some variables according my experiments.

26 comments:

  1. mas , masih lom pernah nyoba shorewall

    kalo config ini

    REDIRECT loc 3127 tcp www -

    di iptables gimana ya ??

    ReplyDelete
    Replies
    1. iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j REDIRECT --to-port 3127

      Delete
    2. eth1 sebagai eth internet ya??
      terus yang

      REDIRECT loc 3129 tcp 443 -

      itu eth local network ya , dalam settingan mas roel eth0??

      gimana sih mas bedakan , user/client akses dr cache squid kita atau direct ke wb tsb tp via proxy kita ?? sorry kalo membingungkan

      Delete
    3. eth1 itu ethernet ke arah users / client
      eth0 itu ethernet ke arah internet.
      posting ini berhubungan dengan posting sebelumnya. baca posting sebelumnya, http://myconfigure.blogspot.com/2013/05/how-to-make-tproxy-transparent-proxy-in.html...
      pada posting sebelumnya jelas, konfigurasi shorewall dari awal, kalo di posting ini, saya jelaskan yang berubah saja.
      jika sudah transparent proxy, users otomatis ke internet tapi harus melewati squid dulu, cek dengan tail -f /var/log/squid/access.log

      Delete
    4. sebelumnya saya udah setting ngikutin guide ini

      http://www.cyberciti.biz/tips/linux-setup-transparent-proxy-squid-howto.html...

      http udah ok, kemudian ikutin petunjuk anda. https masih error..misal ke gmaill selalu muncul back to safety.

      setalah saya rubah


      -A PREROUTING -i eth1 (awalnya eth0) -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3130
      -A PREROUTING -i eth0 (awalnya eth1) -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3127

      koq malah bisa ?
      eth0 arah ke client, eth1 ke arah internet

      makanya saya tkutnya direct lgsung, mohon pencerahannya.
      kalo gunakan shorewall , rule2 yg sudah ada di iptables sudah tidak berlaku ??

      Delete
    5. sudah saya liat ke site tersebut... kalo saya tidak salah tangkap artinya dari lan dan internet bisa menggunakan squid itu...
      coba saja cek dengan tail -f /var/log/squid/access.log ada terlihat trafik tidak...
      jika menggunakan shorewall, shorewall otomatis membuat rule2 iptables...

      Delete
    6. "lan dan internet bisa menggunakan squid itu..."
      seharusnya cukup lan bisa ke squid tsb ??

      kalo saya tail ada trafik sih,tp untuk misal gooogle
      ga ada trafik (apa salah liat ya), apa karena dia ambil dr cache?
      walaupun ambil dr cache tetep ada trafik ya ??

      yg prerouting post saya seblumnya emang bener begitu (sehingga bisa aksess https) atau memang ada keanehan ??

      mater nuwun

      Delete
    7. ya cukup lan, lah... karena kalo internet bisa menggunakan squid yang anda buat, artinya bandwidth upload anda diambil. kecuali memang anda punya bandwidth besar, dan ikhlas ngasi. trafik ambil dari cache atau tidak, akan tetap terlihat di access.log.
      apakah anda sudah menginstall openssl ?? dan mengikut langkah2 di posting ini ?? intinya, jika https bisa menggunakan transparent proxy, adalah jika anda ke situs facebook, atau situs email, maka akan ada tampilan scurity, seperti gambar di atas,...
      cek lagi iptablesnya, intinya -A PREROUTING -i "eth yg menuju lan".....

      Delete
    8. sebelumnya saya udah ikutin setting shorewall mas di

      http://myconfigure.blogspot.com/2013/05/how-to-make-tproxy-transparent-proxy-in.html

      http dan https udah ok, kemudian ikutin petunjuk yang di

      http://myconfigure.blogspot.com/2013/05/how-to-make-transparent-proxy-https-ssl.html

      tetapi hasilnya https langsung error..misal ke https://www.gmail.com atau https://login.yahoo.com selalu muncul unable to connect.

      berikut script shorewall saya :

      rules
      REDIRECT loc 3128 tcp www
      REDIRECT loc 3129 tcp 443
      ACCEPT $FW net tcp 53,80,443
      ACCEPT $FW net udp 53

      ACCEPT loc net tcp 53
      ACCEPT loc net udp 53

      ===================================================

      policy
      loc $FW REJECT
      loc net REJECT

      $FW net REJECT

      net $FW REJECT
      net all DROP info

      all all REJECT info

      =============================================================

      & ini script Squid saya :
      http_port 3128 transparent
      https_port 3129 transparent ssl-bump cert=/etc/squid/proxy.pem key=/etc/squid/proxy.pem

      #Common User
      acl my_network src 192.168.25.0/24

      #Using http_access
      http_access allow my_network

      cache_mem 64 MB
      cache_swap_high 100%
      cache_swap_low 80%

      #Speedy Dns
      dns_nameservers 8.8.8.8 192.168.42.8 192.168.42.12

      maximum_object_size 2048 KB
      minimum_object_size 4 KB

      visible_hostname BAPGW02
      cache_mgr support@binaartha.com

      forwarded_for off
      ssl_bump allow all
      always_direct allow all

      ===========================================================

      disini saya memakai Squid 3.1.18 and shorewall 3.4, apakah ada masalah yang harus diperbaiki? baik dari segi script atau dari segi software (squid and shorewall yang saya pakai tidak stabil)? mohon pencerahannya..terimakasih..wisnu

      Delete
    9. Sepertinya sudah tidak bisa lagi mencache https/ssl. Bukan kesalahan dari anda, tapi dari yang punya situs, tidak mau dicache... sabar...

      Delete
  2. squid 3.3.9 rpm el6 ada di repohttp://www1.ngtech.co.il/rpm/centos/6/

    ReplyDelete
  3. ==> /var/log/squid/access.log <==
    1384442530.725 577 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442530.995 225 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442531.252 233 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442531.505 220 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442531.751 228 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442532.017 232 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442532.278 227 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442532.529 228 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442532.790 224 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442533.042 227 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442533.294 227 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442533.539 223 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html
    1384442533.793 227 172.16.20.2 TCP_MISS/302 614 GET http://www.google.com.br/ - DIRECT/200.172.62.25 text/html

    ReplyDelete
  4. Selamat siang mas,

    Saya sudah mengikuti step-by-step how to diatas, untuk redirect port 80 ke 3127 sudah dapat berjalan dengan baik, hanya utk https redirect 443 ke 3129 terdapat error pada access.log :

    Nov 22 04:24:03 141 172.16.197.2 TCP_MISS/502 4252 GET http://www.facebook.com/ - HIER_DIRECT/31.13.79.1 text/html

    apakah ada petunjuk lainnya lagi mas?

    Terima kasih

    ReplyDelete
  5. errornya apa ?? di file access log... harus ada log https://.... artinya sudah berhasil mencache...

    ReplyDelete
    Replies
    1. Error yang muncul "Read Error" dan kadang "Connection reset by peer"
      dan di access.log tdk muncul https:// melainkan sewaktu kita membuka situs misalnya
      https://www.facebook.com/ yang muncul di access.log adalah :

      Nov 22 04:24:03 141 172.16.197.2 TCP_MISS/502 4252 GET http://www.facebook.com/ - HIER_DIRECT/31.13.79.1 text/html

      Apakah harus dibuat TPROXY seperti pada tuts http://myconfigure.blogspot.com/2013/05/how-to-make-tproxy-transparent-proxy-in.html karena sy hanya menambahkan dibagian shorewall untuk rule nya saja :

      REDIRECT loc 3127 tcp www -
      REDIRECT loc 3129 tcp 443 -

      Sedangkan pada file shorewall -> tcrules dan providers tdk tambahkan (TPROXY & DIVERT).

      Mohon petunjuknya, terima kasih.

      Delete
    2. oh iya sebagai tambahan sy menggunakan SQUID3.HEAD release tgl kmrn...apakah itu penyebabnya ya? atau harus menggunakan v3.3.5.

      saya coba tuts TPROXY http://myconfigure.blogspot.com/2013/05/how-to-make-tproxy-transparent-proxy-in.html juga tidak berhasil... setelah selesai diconfig test buka web keluar "page can't not display"

      OS yang saya gunakan ClearOS 6.5.

      untuk package2 dan module yang diperlukan sudah terinstall seluruhnya, serta untuk shorewall jg udah berjalan dengan smooth :)

      Terima kasih.

      Delete
    3. sudah install openssl kah ?? dan melakukan langkah-langkah seperti di atas...
      kalo 3.3.x atau 3.2.x, gunakan transparent proxy biasa saja... jangan tproxy...
      coba di baca hati-hati lagi... tentang posting saya di atas.. karena memang ada link yang lain... dan itu harus diikut/dibaca juga...

      Delete
    4. openssl sudah terinstall juga dan sudah bisa jalankan perintah ini :
      openssl req -new -newkey rsa:1024 -days 365 -nodes -x509 -keyout myconfigure.pem -out myconfigure.pem

      transparent proxy = intercept itu ya mas?

      baik mas...sy coba reinstall lagi menggunakan squid 3.1/3.5 :)

      Terima kasih

      Delete
  6. Mas Roel Ngeri terima kasih buat tutorialnya,, dari beberapa tutorial squid cuma ini yang sukses buat blok httpsnya.. Untuk ssl google, youtube, and login yahoo yang eror tinggal tambahin nama domain server di configurasi IPnya saja.

    Terima Kasih

    ReplyDelete
    Replies
    1. thx, walaupun memang masih banyak kekurangan,.. tampilan facebook masih berantakan... :)

      Delete
  7. Bang Rull emang TOP !!!!

    tapi bang Rull, punya saya login google error

    access log :
    TCP_MEM_HIT/200 22598 POST https://accounts.google.com/ServiceLoginAuth - HIER_NONE/- text/html

    gimana cara except ni HIT.
    @book satu lhoe udah fix ya?
    bantuin gua dong...

    ReplyDelete
  8. problem resolved

    by add
    acl https_login url_regex ^https://accounts.*

    ada saran yang lebih baik ?

    ReplyDelete
  9. bang rul,
    sertifikatnya harus manual ya d kliennya?
    supaya otomatis gmn bang rul?

    ReplyDelete
    Replies
    1. ya betul, masih manual... yang otomatis...mmm masih belum ketemu cara yang tepat...

      Delete
  10. The best things about Clixsense's GPT PROGRAM/}GPT Program:
    1. SURVEYS: 50+ 5-40 minute surveys paying out $0.50-$2.50 per survey.
    2. 12 Offer Walls - Get $0.5-$20 per offer.
    3. MICRO TASKS - Complete 1,000's of small tasks from many companies.

    ReplyDelete