Software/Wiki2mindmap

This program draws a mindmap from the mediawiki at www.staerk.de/thorsten. This program prerequesites Spider. Thanks to henna for his ideas.

Programmer's todo

• enable other mediawikis
• get along with topics containing a slash

#!/bin/bash
# wiki2mindmap aka wikimap
# draws a mindmap from a mediawiki.

function get() {
  if grep -E "$1" got.log
  then echo "This was here already"
  else
    wget $1
    echo $1 >> got.log
    addcontent $1
  fi
}
 
function addcontent() {
  # write that stuff into the mindmap
  echo "Entering addcontent $1"
  export filename=$(echo $1 | sed "s;.*/\(.*\);\1;")
  spider $filename 2>&1 | sed "s;%C3%BC;ü;" | sed "s;%C3%B6;ö;" | grep -v "&" | grep -v "Current_events" | grep -v ":" |grep -v ^# \
  | sed "s/%/_/g" | sed "s;.*/\(.*\);\1;"| (while read a; do if [ "x$filename" != "x$a" ]; then echo "\"$filename\" -> \"$a\"" >> graph.dot; fi; done)

 
  # get that stuff
  spider $filename 2>&1 | sed "s;%C3%BC;ü;" | sed "s;%C3%B6;ö;" | grep -v "&" | grep -v "Current_events" | grep -v ":" | grep -v ^# \
  | (while read a; do get $hosturl$a; done)
}
 
if (which dot && which konqueror && which spider); then
  echo "Start URL ? "
  read starturl
  if [ x$starturl = x ]; then export starturl="http://developernew.kde.org"; fi
  hosturl=$(echo $starturl|sed "s;\(http://[^/]*\)/.*;\1;")
  echo $hosturl

  rm got.log
  echo "# to create a ps file use 'dot -Tps -o graph.ps graph.dot' (dot it part of the graphviz package)" > graph.dot
  echo "digraph \"Wikimap\" { " >> graph.dot 
 
  get $starturl
  
  echo "}" >> graph.dot
  cat graph.dot | uniq > graph.dot
  sed -i "s/.*->.*Main_Page\"//" graph.dot
  dot -Tps -o graph.ps graph.dot 
  konqueror graph.ps
else
  echo "You do not have at least one of the following installed:"
  echo "dot (from the graphviz package)"
  echo "konqueror"
  echo "spider (from www.staerk.de/thorsten)"
fi

other ideas

get all files

Allpages

Make use of the fact that Allpages does not know more than two levels of recursion:

wget -r --level=2 http://www.staerk.de/thorsten/index.php/Special:Allpages
/bin/spider Zcat | grep "^/wiki" | grep -v ":" | grep -vi "Main_Page"            
ls --ignore="*:*"|less

PopPages

like this, you can show all pages on one page: http://wiki.linuxquestions.org/index.php?title=Special:Popularpages&limit=4000&offset=0

draw the dot file

for file in $(ls --ignore="*:*"); do /bin/spider $file | grep "^/wiki" | grep -v ":" | grep -vi "Main_Page" | sed "s;^/wiki/;;" | while read file2; do echo "$file -> $file2" ; done; done