From ThorstensHome
Revision as of 08:41, 11 January 2010 by WikiSysop (Talk)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

This program draws a mindmap from the mediawiki at This program prerequesites Spider. Thanks to henna for his ideas.

This software is experimental. You cannot use it, but you can get some ideas for software development from it.


Programmer's todo

  • enable other mediawikis
  • get along with topics containing a slash
# wiki2mindmap aka wikimap
# draws a mindmap from a mediawiki.

function get() {
  if grep -E "$1" got.log
  then echo "This was here already"
    wget $1
    echo $1 >> got.log
    addcontent $1
function addcontent() {
  # write that stuff into the mindmap
  echo "Entering addcontent $1"
  export filename=$(echo $1 | sed "s;.*/\(.*\);\1;")
  spider $filename 2>&1 | sed "s;%C3%BC;ü;" | sed "s;%C3%B6;ö;" | grep -v "&" | grep -v "Current_events" | grep -v ":" |grep -v ^# \
  | sed "s/%/_/g" | sed "s;.*/\(.*\);\1;"| (while read a; do if [ "x$filename" != "x$a" ]; then echo "\"$filename\" -> \"$a\"" >>; fi; done)

  # get that stuff
  spider $filename 2>&1 | sed "s;%C3%BC;ü;" | sed "s;%C3%B6;ö;" | grep -v "&" | grep -v "Current_events" | grep -v ":" | grep -v ^# \
  | (while read a; do get $hosturl$a; done)
if (which dot && which konqueror && which spider); then
  echo "Start URL ? "
  read starturl
  if [ x$starturl = x ]; then export starturl=""; fi
  hosturl=$(echo $starturl|sed "s;\(http://[^/]*\)/.*;\1;")
  echo $hosturl

  rm got.log
  echo "# to create a ps file use 'dot -Tps -o' (dot it part of the graphviz package)" >
  echo "digraph \"Wikimap\" { " >> 
  get $starturl
  echo "}" >>
  cat | uniq >
  sed -i "s/.*->.*Main_Page\"//"
  dot -Tps -o 
  echo "You do not have at least one of the following installed:"
  echo "dot (from the graphviz package)"
  echo "konqueror"
  echo "spider (from"

other ideas

get all files


Make use of the fact that Allpages does not know more than two levels of recursion:

wget -r --level=2
/bin/spider Zcat | grep "^/wiki" | grep -v ":" | grep -vi "Main_Page"            
ls --ignore="*:*"|less


like this, you can show all pages on one page:

draw the dot file

for file in $(ls --ignore="*:*"); do /bin/spider $file | grep "^/wiki" | grep -v ":" | grep -vi "Main_Page" | sed "s;^/wiki/;;" | while read file2; do echo "$file -> $file2" ; done; done