Squid - squid, DNS dnsserver, ftp ftpget, . squid , dnsserver-, , DNS . DNS.
Squid ARPA Harvest. http://harvest.cs.colorado.edu/
- Harvest. Squid , .
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
, : squid-faq@nlanr.net.
% gzip -dc squid-x.y.z-src.tar.gz | tar xvf -
,
% cd squid-x.y.z
% ./configure
% make all
% make install
GNU C (gcc) . ANSI C, . GNU C ftp://prep.ai.mit.edu/pub/gnu/. . --prefix . /usr/local/squid. :
% cd squid-x.y.z
% ./configure --prefix=/some/other/directory/squid
/usr/local/squid/bin/RunCache &
% client http://www.netscape.com/ > test
HTTP - . :url_get, ftp://ftp.pasteur.fr/pub/Network/url_get/, echoping, ftp://ftp.pasteur.fr/pub/Network/echoping/. access.log cache.log.
cd squid-1.1.x patch < /tmp/fixes.patch 'src', :
cd squid-1.1.x/src patch < /tmp/fixes.patch patch - , GNU FTP site.
, squid.conf childcache.example.com , :
# squid.conf - On the host: childcache.example.com # # Format is: hostname type http_port udp_port # cache_host parentcache.example.com parent 3128 3130 cache_host childcache2.example.com sibling 3128 3130 cache_host childcache3.example.com sibling 3128 3130cache_host_domain , :
# squid.conf - On the host: sv.cache.nlanr.net # # Format is: hostname type http_port udp_port # cache_host electraglide.geog.unsw.edu.au parent 3128 3130 cache_host cache1.nzgate.net.nz parent 3128 3130 cache_host pb.cache.nlanr.net parent 3128 3130 cache_host it.cache.nlanr.net parent 3128 3130 cache_host sd.cache.nlanr.net parent 3128 3130 cache_host uc.cache.nlanr.net sibling 3128 3130 cache_host bo.cache.nlanr.net sibling 3128 3130 cache_host_domain electraglide.geog.unsw.edu.au .au cache_host_domain cache1.nzgate.net.nz .au .aq .fj .nz cache_host_domain pb.cache.nlanr.net .uk .de .fr .no .se .it cache_host_domain it.cache.nlanr.net .uk .de .fr .no .se .it cache_host_domain sd.cache.nlanr.net .mx .za .mu .zm, pb.cache.nlanr.net it.cache.nlanr.net uk, de, fr, no, se it, sd.cache.nlanr.net mx, za, mu zm, cache1.nzgate.net.nz au, aq, fj, nz.
cache_announce 24 announce_to sd.cache.nlanr.net:3131 : NLANR. NLANR , NLANR.
(, ). HTTP . 80 ( ), , HTTP "" HTTP ( ). ( ).
web , HTTP , . ICP web .
Squid . , HTTP URL , .
" " , . , web- . , . , .
Squid Harvest CERN . httpd , web- ( 80 ), httpd ( 81 ).
web URL 81 httpd. , HTML GIF-, httpd ( 81 ) - , cgi-bin . , web-.
, squid httpd- , . . Squid httpd- , httpd_accel_with_proxy on squid.conf.
inside_firewall squid.conf . :
inside_firewall example.com
:
inside_firewall example.com example.org example.net
inside_firewall . . :
dnsserver , squid . dnsserver-, squid, . , dnsserver , . , dnsserver .
socks5, Squid. , -Dbind=SOCKSbind etc -lsocks .
Netscape Navigator. Netscape Navigator JavaScript http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/proxy-live.html
JavaScript Oskar Pearson:
//We (www.is.co.za) run a central cache for our customers that they //access through a firewall - thus if they want to connect to their intranet //system (or anything in their domain at all) they have to connect //directly - hence all the "fiddling" to see if they are trying to connect //to their local domain. //Replace each occurrence of company.com with your domain name //and if you have some kind of intranet system, make sure //that you put it's name in place of "internal" below. //We also assume that your cache is called "cache.company.com", and //that it runs on port 8080. Change it down at the bottom. //(C) Oskar Pearson and the Internet Solution (http://www.is.co.za) function FindProxyForURL(url, host) { //If they have only specified a hostname, go directly. if (isPlainHostName(host)) return "DIRECT"; //These connect directly if the machine they are trying to //connect to starts with "intranet" - ie http://intranet //Connect directly if it is intranet.* //If you have another machine that you want them to //access directly, replace "internal*" with that //machine's name if (shExpMatch( host, "intranet*")|| shExpMatch(host, "internal*")) return "DIRECT"; //Connect directly to our domains (NB for Important News) if (dnsDomainIs( host,"company.com")|| //If you have another domain that you wish to connect to //directly, put it in here dnsDomainIs(host,"sistercompany.com")) return "DIRECT"; //So the error message "no such host" will appear through the //normal Netscape box - less support queries :) if (!isResolvable(host)) return "DIRECT"; //We only cache http, ftp and gopher if (url.substring(0, 5) == "http:" || url.substring(0, 4) == "ftp:"|| url.substring(0, 7) == "gopher:") //Change the ":8080" to the port that your cache //runs on, and "cache.company.com" to the machine that //you run the cache on return "PROXY cache.company.com:8080; DIRECT"; //We don't cache WAIS if (url.substring(0, 5) == "wais:") return "DIRECT"; else return "DIRECT"; }
% setenv http_proxy http://mycache.example.com:3128/
% setenv gopher_proxy http://mycache.example.com:3128/
% setenv ftp_proxy http://mycache.example.com:3128/
Lynx lynx.cfg. Lynx . :
http_proxy:http://mycache.example.com:3128/ ftp_proxy:http://mycache.example.com:3128/ gopher_proxy:http://mycache.example.com:3128/
Microsoft Netscape JavaScript. , MSIE 3.0a Windows 3.1 Windows NT 3.51 (, 3.01 build 1225 Windows 95 NT 4.0, ).
MSIE , Options View. Advanced Automatic Configuration. URL JavaScript. MSIE. MSIE JavaScript .
, , . .
Warning: this technique has several significant shortcomings!
# # Code maturity level options # CONFIG_EXPERIMENTAL=y # # Networking options # CONFIG_FIREWALL=y # CONFIG_NET_ALIAS is not set CONFIG_INET=y CONFIG_IP_FORWARD=y # CONFIG_IP_MULTICAST is not set CONFIG_IP_FIREWALL=y # CONFIG_IP_FIREWALL_VERBOSE is not set CONFIG_IP_MASQUERADE=y CONFIG_IP_TRANSPARENT_PROXY=y CONFIG_IP_ALWAYS_DEFRAG=y # CONFIG_IP_ACCT is not set CONFIG_IP_ROUTER=yhttp://www.xos.nl/linux/ipfwadm/ ipfwadm . Ipfwadm . /etc/rc.d/rc.inet1 (Slackware) . . , forwarding (masquerading).
/etc/rc.d/rc.firewall:
#!/bin/sh # rc.firewall Linux kernel firewalling rules FW=/sbin/ipfwadm # Flush rules, for testing purposes for i in I O F # A # If we enabled accouting too do ${FW} -$i -f done # Default policies: ${FW} -I -p rej # Incoming policy: reject (quick error) ${FW} -O -p acc # Output policy: accept ${FW} -F -p den # Forwarding policy: deny # Input Rules: # Loopback-interface (local access, eg, to local nameserver): ${FW} -I -a acc -S localhost/32 -D localhost/32 # Local Ethernet-interface: # Redirect to Squid proxy server: ${FW} -I -a acc -P tcp -D default/0 80 -r 80 # Accept packets from local network: ${FW} -I -a acc -P all -S localnet/8 -D default/0 -W eth0 # Only required for other types of traffic (FTP, Telnet): # Forward localnet with masquerading (udp and tcp, no icmp!): ${FW} -F -a m -P tcp -S localnet/8 -D default/0 ${FW} -F -a m -P udp -S localnet/8 -D default/0 80 . - :
IP firewall input rules, default policy: reject type prot source destination ports acc all 127.0.0.1 127.0.0.1 n/a acc/r tcp 10.0.0.0/8 0.0.0.0/0 * -> 80 => 80 acc all 10.0.0.0/8 0.0.0.0/0 n/a acc tcp 0.0.0.0/0 0.0.0.0/0 * -> *squid.conf:
http_port 80 icp_port 3130 httpd_accel virtual 80 httpd_accel_with_proxy on, virtual !
Windows 95 Microsoft Internet Explorer 3.01 Netscape Communicator .
squid 80 . :
${FW} -I -a rej -P tcp -S localnet/8 -D dec/32 80 IP firewall input rules, default policy: reject type prot source destination ports acc all 127.0.0.1 127.0.0.1 n/a rej tcp 10.0.0.0/8 10.0.0.1 * -> 80 acc/r tcp 10.0.0.0/8 0.0.0.0/0 * -> 80 => 80 acc all 10.0.0.0/8 0.0.0.0/0 n/a acc tcp 0.0.0.0/0 0.0.0.0/0 * -> *: , URL , . , DNS .
DNS ( IMHO) .
access.log, :
Host Ident - [D/M/Yr:H:M:S TZ] "Method URL" Status Sizeaccess.log, Squid 1.0 :
Time Elapsed Host Status/HTTP/Hier_Status Size Method URLaccess.log, Squid 1.1 :
Time Elapsed Host Status/HTTP Size Method URL Ident Hier_Status/Hier_Hosthierarchy.log, Squid 1.0:
[D/M/Yr:H:M:S TZ] URL Hier_Status Hier_Hostlog:
kill -USR1 `cat /usr/local/squid/logs/squid.pid`
: logfile_rotate squid.conf log . logfile_rotate . logfile_rotate , log . logfile_rotate crontab squid ' SIGUSR1, :
0 0 * * * /bin/kill -USR1 `cat /usr/local/squid/logs/squid.pid`
, log, cache_dir . , Squid. .
sort -r -n +4 -5 access.log | awk '{print $5, $7}' | head -25
, , log cache_dir.
, web , SIGHUP, .
web , - URL:
http://www.example.com/Squid/cgi-bin/cachemgr.cgi
Protection MGR-PROT { Mask @(workstation.example.com) }, IP , . . .
:
Protect /Squid/* MGR-PROT Exec /Squid/cgi-bin/*.cgi /usr/local/squid/bin/*.cgiMGR-PROT, .
ScriptAlias /Squid/cgi-bin/ /usr/local/squid/cgi-bin/ScriptAlias /usr/local/squid/bin Squid.
, -. access.conf Apache, squid.conf. access.conf, :
<Location /Squid/cgi-bin/cachemgr.cgi> order deny,allow deny from all allow from workstation.example.com </Location>, .
, cachemgr.cgi . access.conf:
<Location /Squid/cgi-bin/cachemgr.cgi> AuthUserFile /path/to/password/file AuthGroupFile /dev/null AuthName User/Password Required AuthType Basic <Limit GET> require user cachemanager </Location>Apache htpasswd .
acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl all src 0.0.0.0/0.0.0.0:
http_access deny manager !localhost http_access allow allACL -, squid cache_object . :
telnet mycache.example.com 3128
GET cache_object://mycache.example.com/info HTTP/1.0
, cache_object, , , - .
, , cachemgr.cgi localhost. :
acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl example src 123.123.123.123/255.255.255.255 acl all src 0.0.0.0/0.0.0.0123.123.123.123 IP web . :
http_access deny manager !localhost !example http_access allow all, web , squid. , - squid web , . , web - , IP web , cachemgr.cgi example .
squid.conf SIGHUP squid'.
HOST_OPT = # -DCACHEMGR_HOSTNAME="getfullhostname()"web cachemgr.cgi , Squid #. web - , :
HOST_OPT = -DCACHEMGR_HOSTNAME=\"mycache.example.com\"cachemgr.cgi.
squid , , ! - , squid.
Flags:
N IP , hostname.
IP , IP cache.
Memory usage for squid via mallinfo(): Total space in arena: 94687 KB Ordinary blocks: 32019 KB 210034 blks Small blocks: 44364 KB 569500 blks Holding blocks: 0 KB 5695 blks Free Small blocks: 6650 KB Free Ordinary blocks: 11652 KB Total in use: 76384 KB 81% Total free: 18302 KB 19% Meta Data: StoreEntry 246043 x 64 bytes = 15377 KB IPCacheEntry 971 x 88 bytes = 83 KB Hash link 2 x 24 bytes = 0 KB URL strings = 11422 KB Pool MemObject structures 514 x 144 bytes = 72 KB ( 70 free) Pool for Request structur 516 x 4380 bytes = 2207 KB ( 2121 free) Pool for in-memory object 6200 x 4096 bytes = 24800 KB ( 22888 free) Pool for disk I/O 242 x 8192 bytes = 1936 KB ( 1888 free) Miscellaneous = 2600 KB total Accounted = 58499 KBmallinfo() , r 94M. , top (97M).
94M, 81% (76M) . , malloc(3) .
76M , 58.5M (76%). malloc(3).
Meta Data , . 45% StoreEntry URL . 42% , (Pool for in-memory object).
squid.conf. 1.0, : , . Pool for in-memory object, 1/2 cache_mem. Pool for disk I/O 200. MemObject Request 1/8 FD_SETSIZE.
, 'http', 'ftp' 'gopher' . cache_mem. cache_mem , . Squid memory_pools off .
:
============================================================================== IP Cache Contents: Hostname Flags lstref TTL N [IP-Number] gorn.cc.fh-lippe.de C 0 21581 1 193.16.112.73 lagrange.uni-paderborn.de C 6 21594 1 131.234.128.245 www.altavista.digital.com C 10 21299 4 204.123.2.75 204.74.103.37 204.123.2.66 204.123.2.69 2/ftp.symantec.com DL 1583 -772855 0 Flags: C --> D --> N --> L --> lstref: TTL: Time-To-Live ( ) N: ============================================================================== FQDN Cache Contents: IP-Number Flags TTL(?) N Hostname] 130.149.17.15 C -45570 1 andele.cs.tu-berlin.de 194.77.122.18 C -58133 1 komet.teuto.de 206.155.117.51 N -73747 0 Flags: C --> D --> N --> L --> TTL: Time-To-Live N:
http_accel_with_proxy on
, ACL. access.log squid.conf.
local_domain . . , cache_stoplist http_stop ( ).
Linux, filehandle.patch.linux Michael O'Reilly <michael@metal.iinet.net.au>.
Solaris, /etc/system:
set rlim_fd_max = 4096
set rlim_fd_cur = 1024
#define SQUID_FD_SETSIZE include/config.h , rlim_fd_max. 4096.
Solaris select(2) 1024 , src/Makefile $(USE_POLL_OPT). squid.
FreeBSD ( Torsten Sturm <torsten.sturm@axis.de>):
cd squid-1.1.x make realclean ./configure --prefix=/usr/local/squid make
options DFLDSIZ=67108864 # 64 meg default max data size (was 16) options MAXDSIZ=134217728 # 128 meg max data size (was 64).
Digital UNIX, /etc/sysconfigtab ...
proc: per-proc-data-size=1073741824, csh, limit ...
/etc/sysconfigtab , limit - .
97/01/23 22:31:10| Removed 1 of 9 objects from bucket 3913 97/01/23 22:33:10| Removed 1 of 5 objects from bucket 4315 97/01/23 22:35:40| Removed 1 of 14 objects from bucket 6391log , , squid cache_swap_high.
cache information cachemgr.cgi :
Storage LRU Expiration Age: 364.01 days, , . LRU Expiration Age reference_age .
FATAL: Don't run Squid as root, set 'cache_effective_user'!, cache_effective_user nobody, . , Squid cache_effective_user.
UID nobody 65535 65534.
( ), "Properties" , "directories", "Directory listing style." "Unix" type, "MS-DOS" type.
--Oskar Pearson <oskar@is.co.za>(1) , DNS. -, . IP DNS, Squid 'udp_outgoing_address'.
:
# (squid.conf ) # udp_outgoing_address proxy.parent.com # ( squid.conf) # cache_host proxy.parent.com parent 3128 3130(2) ICP . , Squid , . , log . , , , , .
ICP . squid , ICP , ICP "HIT" ("") "MISS" (""). MISS.
ICP TCP . ICP UDP. Squid ICP.
Squid /, DNS . dnsserver DNS, squid`.
, ftpput.
/ , squid , , . , Internet, , . , "".
single_parent_bypass ICP , ( , , ?)
, http://squid.nlanr.net/Squid/Devel/todo.html.
http://squid.nlanr.net/Squid/Devel/.
Last-modified: Tue, 01 Dec 1998 21:18:05 GMT