Tap the Usenet
A powerful package is Rich $alz's "InterNetNews", say INN. It isn't easy to master the INN. Who made the sendmail may be happy: it becomes more difficult...!
INN would be able to fetch and send news by itself, but we will get some cooperation from a more intelligent package: newsflash and postit.
Again I'll describe it depending on the S.u.S.E. 4.2 Linux distribution. Here INN is easily installable in the /usr/lib/news directory (with the "YaST" tool). By the way you can find here a (sorry, not very useful) overview: /usr/lib/news/README.linux, and INN is pre-installed (if marked in YaST), but all the Documentation is missing. You can get the documentation, if you send an email to majordomo@warren.mentorg.com
Subject: empty sendme faq-inn-1 sendme faq-inn-2 sendme faq-inn-3Then you will receive a long FAQ describing how INN works and how to fix many problems. You also should read the tutorial (in German language) news-0.3.ps by Markus Dickebohm. It describes in a very good way how to install a INN/newsflash/postit system. Have a look at this server: Maybe there are newer tutorials available. And you could also load down the packages newsflash and postit (see below: "Give and Take").
nntp stream tcp nowait root /usr/sbin/tcpd /usr/sbin/in.nntpdYou must add a "#" in the first column to make it a comment, otherwise innd will not start (I needed several days to notice that). INN does not run together with C News or any other news daemon, too. It isn't necessary, by the way.
There is one thing you also should do. If INN does not like something it writes an email to "usenet@localhost". You must tell your system that "usenet" is a user of your system - some other mail addresses too. You do it by editing /etc/aliases. Add the following lines (perhaps you have to create a new file):
/etc/aliases news: root newsadm: news newsadmin: news usenet: newsType in to tell sendmail the new aliases:
root:# newaliases root:# _Now all emails concerning your Usenet system come to root.
But before we start configuring INN we have to edit the file /etc/hosts and to insert one entry for the news server. Take the address from the database or call the provider's hotline. Add:
/etc/hosts (cutting) <IP address> <name_of_mail_server> getmail
/var/lib/news/inn.conf (Slackware: /usr/lib/news/inn.conf) # Basic installation of INN server: your.computer organization: as you like fromhost: Email_adresse@Host.Computer # for posting headerThomas Schoepf <schoepf@informatik.tu-muenchen.de> wrote me: if there appears a error message like ME: cant GetConfigValue pathhost... while starting INN, you should add the line pathhost: <Host.Fuer.News.Path>.
/var/lib/news/hosts.nntp (Slackware: /usr/lib/news/hosts.nntp) # Which site feeds our newsreader? localhost: Rechner.name: /var/lib/news/nnrp.access (Slackware: /usr/lib/news/nnrp.access) # Which computers may tap us? localhost:Read Post:::* (none):Read Post:::* your.computer:Read Post:::* /var/lib/news/newsfeeds (Slackware: /usr/lib/news/newsfeeds) # Where do the news go? # We don't feed our news to anywhere. ME\ :*:: # for NOV database @overview:*:Tc,WO:/usr/lib/news/bin/overchan # Feed the LRZ with all we have: get/getlist\ :*:Ap,Tf,Wnm:getPlease refer to the INN documentation for more information. providername is the name the provider writes into the news header in the "Path:" line. You must exacltly write it the way your provider does. If you don't the article negotiation will take much more time.
There are some files missing, which can be fetched from the news server. Therefore you could use the script getlist, which should be part of the INN package. If not, load it via ftp:
unix.hensa.ac.uk/mirrors/yggdrasil.fall95.disk1/usr/lib/news/bin/getlist.Store it in /usr/lib/news/bin and make it executable: "chmod news getlist".
So you can load two files from lrznews using getlist:
root:# /usr/lib/news/bin/getlist -h getnews active >/var/lib/news/active.raw root:# /usr/lib/news/bin/getlist -h getnews newsgroups >/var/lib/news/newsgroups root:# _If you have the Slackware distribution change /var/lib/news against /usr/lib/news.
You even could manage this manually (without "getlist") using telnet:The fresh file active must be edited now. In it you find all the subsribable Usenet groups and the actual posting numbers. Warning: this file is sensitive. It does not like any changes (words per line, number of white spaces or number of digits)! You may not change anything except the things we discuss here!!!root: # telnet getnews 119 >/var/lib/news/active.raw now wait 10 sec. - *no* prompt appears! list active wait again a few seconds quit wait until your own prompt appears (takes some time): root: # telnet getnews 119 >/var/lib/news/newsgroups now wait 10 sec. - *no* prompt appears! list newsgroups wait again a few seconds quit wait until your own prompt appears (takes some time): root: # _
You should do now two things. First, set all posting numbers zo zero. Do it this way:
root:# cd /var/lib/news # Slackware: /usr/lib/news root:# sed 's/ [0-9]* [0-9]* / 0000000000 0000000001 /' active.raw > active root:# _Second, look if there are entries for the following newsgroups. If they are missing add them now.
to 0000000000 0000000001 y junk 0000000000 0000000001 y control 0000000000 0000000001 y
root:# chown root.root /var/run # (not root.uucp!) root:# _(perhaps you need to play something with the access bits)
For testing look into /var/spool/news whether the directories in.coming, news.archive, out.going and over.view are present - if not, create them using mkdir!
Finally you must give all files in /var/lib/news (Slackware: /usr/lib/news and in /var/spool/news the user news:
root:# chown -R news.news /usr/lib/news/* # for Slackware only this one root:# chown -R news.news /var/lib/news/* # if present root:# chown -R news.news /var/spool/news/* # if present root:# _For basic installation call now some assisting programs: makehistory and news.daily. Do it as user news (important!):
root# cd / root# su news -c "/usr/lib/news/bin/makehistory -o" root# su news -c "/usr/lib/news/bin/news.daily" root# _
root:# /usr/lib/news/etc/rc.news Starting innd. root:# telnet localhost 119 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 200 my.computer InterNetNews NNRP server INN version date ready (posting ok). no prompt appears!If this message appears you may breath deeply and say loudly "uff!!!". You can escape here typing in quit.
If the message does not appear, sorry: you need to find the bug, and this may take some time. Let the INN-FAQ help you. Perhaps INN could send you an email to "usenet" - respectively "root", in which you may read what happened. If there is no mail perhaps you can find some hints in the syslog, in /var/log/messages or in the log files in /var/lib/news/log.
The INN installation made some gray hairs onto my head and took more than one month time. I tried to tell you all the mistake possibilities, but I feel I could not cover all of them. The INN is not very cooperative while debugging... But I think I could help you in most cases
Again we have the possibls cases sinpleinit or sysvinit.
/etc/rc.d/rc.M; Sektion "# Initialize the NET subsystem." . /etc/rc.d/rc.inet1 . /etc/rc.d/rc.isdn . /etc/rc.d/rc.inet2 . /usr/lib/news/etc/rc.news # <-- add this line!You also must find the file which is called while system reboot (often this is /etc/rc.d/rc.6). Here you should add before the shutdown commands:
/etc/rc.d/rc.6 echo -n "Shutting down NNTP port:" /usr/lib/news/bin/ctlinnd throttle "shutdown" /usr/lib/news/bin/ctlinnd shutdown "shutdown" [more shutdown commands]If you don't some (harmless) error messages appear while shutdown.
/sbin/init.d/news # (Red Hat: /etc/rc.d/news) #! /bin/sh # Author: Bernhard Hailer <dl4mhk@lrz.uni-muenchen.de>, 24-Jun-96 # /sbin/init.d/news case "$1" in start) echo "Initializing NNTP port. (INN)" /usr/lib/news/etc/rc.news # rc.news wird beim INN mitgeliefert ;; stop) echo -n "Shutting down NNTP port:" /usr/lib/news/bin/ctlinnd throttle "shutdown" /usr/lib/news/bin/ctlinnd shutdown "shutdown" ;; *) echo "Usage: $0 {start|stop}" exit 1 esac exit 0You must set some soft links, because the system needs to know which files have to be executed while starting or stopping:
root:# cd /sbin/init.d/rc2.d # There are other path definitions root:# ln -s ../news K20news # in other Linux distributions! root:# ln -s ../news S20news root:# _Now while booting the INN start-up should appear on the screen - together with the sendmail daemon. Also while shutdown some messages should appear.
/var/cron/tabs/news # (perhaps /var/spool/cron/crontabs/news) PATH=/usr/lib/news/bin:/usr/lib/news/etc:/usr/lib/news:/usr/bin:/usr/sbin 10 07 * * * /usr/lib/news/bin/news.dailyWe tell this now crond:
root:# chown news.news /var/cron/tabs/news root:# crontab -u news /var/cron/tabs/news root:# _Now the script should be called every day at 07:10 a.m.
news.daily does the cleaning and garbage collecting. Old news postings will be erased, and some old log files and so on will be deleted, too. There are two control files which may be interesting:
/var/lib/news/expire.ctl # Slackware: /usr/lib/news/expire.ctl # If posting is erased we will remember its ID 14 days, because # we don't want to receive it again: /remember/:14 # (A)ll postings will be kept min. 3, regularly 10 and max. 30 days # (min. and max., if an expire date is set in the posting) *:A:3:10:30 # some special newsgroups may remain much shorter: junk:A:1:1:2 control:A:1:1:2 /var/lib/news/control.ctl # Slackware: /usr/lib/news/control.ctl # preset: if INN wants to tell something it has to mail it to admin. all:*:*:mail
The program suck is more known, but I choosed the (very compact) programs by Chris Blum, because they are highly configurable. They use the telephone line very efficiently, because they open more than one connection to the news server. So, while the news server may create a news packet, there may take place some discussions about the next packet at the same time.
Load these programs down using ftp at
pople.pc.uni-koeln.de/pub/newsflash-0.99.tgz pople.pc.uni-koeln.de/pub/postit-jun96.tar.gz(please look if there are newer versions!) Store it in the directory /usr/src and extract them:
root:# tar zxfv newsflash-0.99.tgz root:# tar zxfv postit-jun96.tar.gz root:# _
Before compilation you should edit /usr/src/newsflash/config.h and set the path variables correctly for your system (/var/lib/news or /usr/lib/news/newsflash for Slackware). Then:
root:# cd /usr/src/newsflash root:# make dep ... root:# make ... root:# cp newsflash /usr/lib/news/bin root:# cp printdate /usr/lib/news/bin root:# _newsflash needs to know which newsgroups it shall retrieve, so you must tell this in a file: it contains the files listing the desired groups:
/var/lib/news/serverlist # or path defined in config.h! get1 getnews 119 /var/lib/news/packet1 timestamp get2 getnews 119 /var/lib/news/packet2 timestamp get3 getnews 119 /var/lib/news/packet3 timestamp get4 getnews 119 /var/lib/news/packet4 timestampHere four packets are executed, all aim at the news server of the provider. The first word is an ID string, the second is the server defined in /etc/hosts, 119 is the standard news port. The packet description follows, and the last number is a timestamp. You can calculate an active timestamp using the program printdate (included in the newsflash package); the produced 9 digit number tells us how many seconds are gone since January 1st at midnight. To retrieve the new news since the last two days you should subtract about 150000 or so (one day are 86400 seconds) and type in the result as timestamp.
Now we must prepare the packet files we defined above. Newsflash can optimize very good the data transmission parallelizing more than one packet. Here we tell it how to execute the transmission:
/var/lib/news/packet1 # Slackware: /usr/lib/news/newsflash/packet1 de.comm.isdn fido.ger.isdn de.comp.os.linux.networking /var/lib/news/packet2 # Slackware: /usr/lib/news/newsflash/packet2 de.comp.os.linux.networking fido.ger.isdn de.comm.isdn /var/lib/news/packet3 # Slackware: /usr/lib/news/newsflash/packet3 de.comp.os.linux.misc de.comp.os.linux.hardware de.comp.os.linux.x lrz.* /var/lib/news/packet4 # Slackware: /usr/lib/news/newsflash/packet4 lrz.* de.comp.os.linux.x de.comp.os.linux.hardware de.comp.os.linux.miscAs you see the packets 1/2 and 3/4 have the same contents, but in reverse order. So one packet is executed from both ends, and the transmission is optimized. The packets should have the same estimated number of postings.
root:# cd /usr/src/postit root:# make dep ... root:# make ... root:# cp postit /usr/lib/news/bin root:# cp samples/sendnews /usr/lib/news/bin root:# _The last file we copied here (/usr/lib/news/bin/sendnews ) must be edited:
/usr/lib/news/bin/sendnews SPOOLPATH=/var/spool/news OUTGOING=${SPOOLPATH}/out.going BINPATH=/usr/lib/news/bin ${BINPATH}/ctlinnd flush get mv ${OUTGOING}/get ${OUTGOING}/get.postit ${BINPATH}/ctlinnd begin get ${BINPATH}/postit get.postit getnews
If it does not work, you must look for the bug. Read the README. In most cases you need to edit again one of the files hosts.nntp, nnrp.access, newsfeeds etc. and play with the entries.
If it works you can read now the news groups with a news reader like nn, tin or Netscape).
To fetch the news automatically you may use the following script, which is very similar to the email fetcher script.
/sbin/transnews #!/bin/sh # transnews: gets and sends usenet news. # Author: Bernhard Hailer <dl4mhk@lrz.uni-muenchen.de> # Version: 4.0 (27-Jan-97) PATH=/sbin:/usr/sbin:/bin:/usr/bin . /etc/i4l.provider . /etc/i4l.secrets cd / # against PATH=. effect echo ----------------------------------------------------------------------- echo -e "Starting news transfer: \c"; date . /sbin/connect on echo "Checking news server..." set `ping -qc5 -i1 getnews | grep transmitted` if [ $4 -gt 0 ]; then echo "8-) News server online - starting news poll" echo "Sending news..." su news -c "/usr/lib/news/bin/sendnews" echo "Again sleeping 1 sec for PPP handshaking" sleep 1 echo "Fetching news..." su news -c "/usr/lib/news/bin/newsflash -v" echo "Again sleeping 1 sec for PPP handshaking" sleep 1 else echo ":-[ Sorry - news server is down. Try later..." fi . /sbin/isdn off echo -e "Finished news transfer: \c"; date
# Transfer news every night at 04:29 a.m. 29 04 * * * /sbin/transnews 1>> /var/log/news-log 2>> /var/log/news-logAs described in the email chapter cron must be called with the new crontab now. The script scould be started one minute before the email transfer starts (not at the same time!!), so perhaps one tariff unit can be saved. If you would start the transfer at the same time, one of the applications would see a open line where no one is (opening a line takes some time). The connection would be interrupted.
Have Fun in the Usenet!!