Reviewing 3D printers

I've read quite a few 3D printer reviews over the last months and now that I have had the opportunity to collect two months worth of practical experience with a well known 3D printer, I have to conclude that most 3D printer reviews are worthless.

In general, you can disregard magazine reviews.  These are the worst.  The problem with these is that they are usually performed under time pressure and it seems more important for magazines to have a review of 3D printers than to do it well.

Magazine 3D printer reviews at this point are nothing more than glorified unboxing videos in text form and you should probably not take them seriously.   I say "probably", because sooner or later I guess some magazine will start applying serious evaluation methodology.  So far though, I have not seen this.

You will find the most useful reviews in blog postings and forums.  People who own the device and who have used it for at least a few months.   For example Nick Lievendag is pretty good at reviewing 3D printers since he actually uses them to do work.

Here is a brief list of what a proper review should minimally involve:

  • Test the machine for at least a couple of months.  It should have at least 350-500 hours of printing on it before you conclude anything.  If you think this is a lot: this is rather typical load when you do a project.  Prints take a long time so you try to minimize the idle time so that you can get more done per day.  50-60 hours of printing time per week is a relatively moderate load during a project.  You can easily double that if you time things properly and you have 24/7 access to the machine.
  • Test it until something breaks.  Some machines have weak parts.  You need to know which parts fail first and you need to know what it will take to repair the machine.  What is covered by warranty is really not interesting.  Breakdowns cause downtime and if you have to return parts and wait for new parts to turn up, that will cost you valuable time.  Also calculate the cost of running the machine over time given the cost of repairs.  (Some 3D printers need spare parts that add up to more than the printer costs within a year.  I have yet to see a single magazine review that addresses this rather important aspect of 3D printer ownership).
  • Take the machine apart and have someone with technical insight analyze the design and the parts used.  Down to looking at what chips have been used for stepper control, what CPU it uses, what kind of firmware, PSU, technology used for calibration etc.  Also get someone who knows about mechanical design to look at the mechanical design.  Precision, durability etc.  If you are squeamish about taking printers apart because you are afraid to piss off manufacturers, reviewing hardware is not for you.  If they try to make you sign NDA agreements you are, of course, disqualified from reviewing the product.
Taking the product apart is more or less a requirement for a serious Magazine review at this point. For privately owned 3D printers this may not be feasible, so it would be unreasonable to expect this from owners.  But people who make a living reviewing stuff:  if you don't you are not serious about what you do.

If you want to learn about tearing down products, I suggest you watch videos by Dave Jones.  He routinely reviews equipment and a standard part of a thorough review is him taking the product apart and analyzing what is inside.  Here's an example of Dave having a look at the GoPro Hero 2.

And if you are a manufacturer and worry about teardowns:  you need to learn how you can benefit from this.  Dave Jones tore apart a Rigol bench lab power supply a while back.  Which uncovered some design errors in the thermal handling of the product.  Rigol responded to this by fixing the problem, and guess what:  since they have fixed it and since Dave has had a thorough look at the product: people now feel safe when buying it.  Because they know it is a good product and that Rigol deals with problems.

PS: I'm in the process of trying to resolve problems with a 3D printer I use for work.  No magazine review I have read correctly reflects what I am experiencing and what other users I have met through forums are experiencing.  I am giving the manufacturer time to resolve issues.  At the end of my current project I plan to write a blog posting or an article about the product and whether or not the issues got resolved.  If the problems with the machine cannot be resolved and if we can't return it for a refund I hope I'll have time to tear it down and possibly rebuild the machine using different 3D printer components.  We'll see what I have time for.

No comments:

Post a Comment