From ThorstensHome

This program draws a mindmap from the mediawiki at www.staerk.de/thorsten. This program prerequesites Spider. Thanks to henna for his ideas.

Warning

This software is experimental. You cannot use it, but you can get some ideas for software development from it.

Programmer's todo

• enable other mediawikis
• get along with topics containing a slash

#!/bin/bash
# wiki2mindmap aka wikimap
# draws a mindmap from a mediawiki.

function get() {
  if grep -E "$1" got.log
  then echo "This was here already"
  else
    wget $1
    echo $1 >> got.log
    addcontent $1
  fi
}
 
function addcontent() {
  # write that stuff into the mindmap
  echo "Entering addcontent $1"
  export filename=$(echo $1 | sed "s;.*/\(.*\);\1;")
  spider $filename 2>&1 | sed "s;%C3%BC;ü;" | sed "s;%C3%B6;ö;" | grep -v "&" | grep -v "Current_events" | grep -v ":" |grep -v ^# \
  | sed "s/%/_/g" | sed "s;.*/\(.*\);\1;"| (while read a; do if [ "x$filename" != "x$a" ]; then echo "\"$filename\" -> \"$a\"" >> graph.dot; fi; done)

 
  # get that stuff
  spider $filename 2>&1 | sed "s;%C3%BC;ü;" | sed "s;%C3%B6;ö;" | grep -v "&" | grep -v "Current_events" | grep -v ":" | grep -v ^# \
  | (while read a; do get $hosturl$a; done)
}
 
if (which dot && which konqueror && which spider); then
  echo "Start URL ? "
  read starturl
  if [ x$starturl = x ]; then export starturl="http://developernew.kde.org"; fi
  hosturl=$(echo $starturl|sed "s;\(http://[^/]*\)/.*;\1;")
  echo $hosturl

  rm got.log
  echo "# to create a ps file use 'dot -Tps -o graph.ps graph.dot' (dot it part of the graphviz package)" > graph.dot
  echo "digraph \"Wikimap\" { " >> graph.dot 
 
  get $starturl
  
  echo "}" >> graph.dot
  cat graph.dot | uniq > graph.dot
  sed -i "s/.*->.*Main_Page\"//" graph.dot
  dot -Tps -o graph.ps graph.dot 
  konqueror graph.ps
else
  echo "You do not have at least one of the following installed:"
  echo "dot (from the graphviz package)"
  echo "konqueror"
  echo "spider (from www.staerk.de/thorsten)"
fi

other ideas

get all files

Allpages

Make use of the fact that Allpages does not know more than two levels of recursion:

wget -r --level=2 http://www.staerk.de/thorsten/index.php/Special:Allpages
/bin/spider Zcat | grep "^/wiki" | grep -v ":" | grep -vi "Main_Page"            
ls --ignore="*:*"|less

PopPages

like this, you can show all pages on one page: http://wiki.linuxquestions.org/index.php?title=Special:Popularpages&limit=4000&offset=0

draw the dot file

for file in $(ls --ignore="*:*"); do /bin/spider $file | grep "^/wiki" | grep -v ":" | grep -vi "Main_Page" | sed "s;^/wiki/;;" | while read file2; do echo "$file -> $file2" ; done; done