The document discusses the implementation of High-Performance Computing (HPC) using a Raspberry Pi cluster in educational settings, highlighting the author's background, inspirations, and teaching materials. It details the author's personal project, 'Tweety Pi', and various demonstrations utilizing Raspberry Pi for HPC training, alongside challenges and feedback from teaching experiences. Future work includes enhancements in configuration management and public engagement initiatives using HPC for research applications.

What's hot

Similar to Introducing HPC with a Raspberry Pi Cluster

More from inside-BigData.com

Recently uploaded

Introducing HPC with a Raspberry Pi Cluster

• 1.
  Introducing HPC witha Raspberry Pi ClusterColin Sauzé <cos@aber.ac.uk>Research Software EngineerSuper Computing Wales ProjectAberystwyth UniversityA practical use of and good excuse to build Raspberry Pi Clusters
• 2.
  Overview● About Me●Inspirations● Why teach HPC with Raspberry Pi?● My Raspberry Pi cluster● Experiences from teaching● Future Work
• 3.
  About Me● ResearchSoftware Engineer withSupercomputing Wales project– 4 university partnership to supplyHPC systems– Two physical HPCs● PhD in Robotics– Experience with Linux on singleboard computers– Lots of Raspberry Pi projects
• 4.
  Inspiration #1: LosAlamos National Laboratory● 750 node cluster● Test system for softwaredevelopment● Avoid tying up the real cluster
• 5.
  Inspiration #2: WeeArchie/Archlet● EPCC’s Raspberry Pi Cluster● Archie: 18x Raspberry Pi 2’s(4 cores each)● Archlet: smaller 4 or 5 nodeclusters.● Used for outreach demos.● Setup instructions:https://github.com/EPCCed/wee_archletImage fromhttps://raw.githubusercontent.com/EPCCed/wee_archlet/master/images/IMG_20170210_132818620.jpg
• 6.
  Inspiration #3: Swansea’sRaspberry Pi Cluster● 16x Raspberry Pi 3s● CFD demo using a Kinectsensor● Demoed at the SwanseaFestival of Science 2018
• 7.
  Why Teach witha Raspberry Pi cluster?● Avoid loading real clusters doing actual research– Less fear from learners that they might break something● Resource limits more apparent● More control over the environment● Hardware less abstract● No need to have accounts on a real HPC
• 8.
  My Cluster● “TweetyPi”– 10x Raspberry Pi model Bversion 1s– 1x Raspberry Pi 3 ashead/login node– Raspbian Stretch● Head node acts as WiFiaccess point– Internet via phone or laptop
• 9.
  Demo Software● BritishScience Week 2019– Simple Pi with Monte Carlo methods demo– MPI based– GUI to control how many jobs launch and showqueuing● Swansea CFD demo– Needs more compute power– 16x Raspberry Pi 3 vs 10x Raspberry Pi 1● Wee Archie/Archlet Demos– Many demos available●I only found this recently– https://github.com/EPCCed/wee_archie
• 10.
  Making a realisticHPC environment● MPICH● Slurm● Quotas on home directories● NFS mounted home directories● Software modules● Network booting compute nodes
• 11.
  Network booting hack●No PXE boot support on original Raspberry Pi (or Raspberry PiB+ and 2)● Kernel + bootloader on SD card● Root filesystem on NFS– Cmdline.txt contains:● console=tty1 root=/dev/nfs nfsroot=10.0.0.10:/nfs/node_rootfs,vers=3 ro ip=dhcp elevator=deadline rootwait● SD cards can be identical, small 50mb image, easy to replace
• 12.
  Teaching Materials● Basedon Introduction to HPC with Super Computing Wales carpentry stylelesson:– What is an HPC?– Logging in– Filesystems and transferring data– Submitting/monitoring jobs with Slurm– Profiling– Parallelising code, Amdahl’s law– MPI– HPC Best Practice
• 13.
  Experiences from Teaching– STFC SummerSchool● New PhD students in solarphysics– Not registered at universitiesyet, no academic accounts● 15 people each time– 1st time using HPC for many– Most had some Unixexperience● Subset of Super ComputingWales introduction to HPCcarpentry lesson
• 14.
  Feedback● Very Positive●Alot seemed to enjoy playing around with SSH/SCP– First time using a remote shell for some– Others more adventurous than they might have been on a real HPC● Main complaint was lack of time (only 1.5 hours)– Only got as far as covering basic job submission– Quick theoretical run through of MPI and Amdahl’s law– Probably have 3-4 hours of material●Queuing became very apparent– 10 nodes, 15 users– “watch squeue” running on screen during practical parts
• 15.
  Problems● Slurm issueson day 1– Accidentally overwrote a system user when creating accounts● WiFi via Laptop/phone slow– When users connect to the cluster its their internet connection too– Relied on this for access to course notes
• 16.
  Experiences from teaching– SupercomputingWales Training● Approximately 10 people– Mix of staff and research students– Mixed experience levels– All intending to use a real HPC● Simultaneously used Raspberry Pi and real HPC– Same commands run on both● Useful backup system for those with locked accounts● Feedback good– Helped make HPC more tangible
• 17.
  Future Work● Configurationmanagement tool(Ansible/Chef/Puppet/Salt etc) insteadof script for configuration● CentOS/Open HPC stack instead ofRaspbian● Public engagement demo whichfocuses on our research– Analysing satellite imagery– Simulate the monsters from MonsterLab(https://monster-lab.org/)
• 18.
  More Information● Setupinstructions and scripts -https://github.com/colinsauze/pi_cluster● Teaching material -https://github.com/SCW-Aberystwyth/Introduction-to-HPC-with-RaspberryPi● Email me: cos@aber.ac.uk