Run new benchmarks and document costs #70

Activity

1. ehoelzl commented on Dec 4, 2020

   

   ehoelzl

   on Dec 4, 2020

   Last edited by ehoelzl

   ContributorAuthor

   More actions

   @martinjaggi 's comment
   "
   there are many use-cases which need to be run

   • the official task results (light and full goals need to run)
   • scaling as part of official results
   • comparing different hardware such as gcloud GPUs K80,P100,P4,T4,V100 etc
   • GPU vs CPU (though official results will be GPU only)
   • comparing backends (probably no scaling the # workers needed)
   • comparing against pytorch DDP or powerSGD, or adam (again this is different than the official results so has different
   • regression testing compared to old versions of our code, and changes in pytorch, TF say
     depending on what's needed for the scientific report (or blog post), that's what we'll run. ideally the code would be easy to adapt
     "
2. ehoelzl

   added

   mlbench-benchmarks

   on Dec 4, 2020