I'm not going to dunk the offending post in water (affirmations recited in a mirror) but I have been simmering all day about someone saying to another user on here ~"why the hell are you using AI for mechanical engineering? you're vibe-coding aerospace engineering! it's gonna kill someone! AI can be subtly wrong and you're using it to make engineering decisions??"
and setting aside that the use case at hand appeared to be "automating tasks that are done via a GUI to be done via the scripting interface instead," aka making a script to push some buttons for you in a FEM solver, and this is something which is both a completely straightforward use case for an LLM (coding!) and entirely verifiable (did the script work lol) - even setting that aside -
I am still split between a few things and almost all of them are "this is, like, irritatingly ignorant of how anything works."
1) Yes, AI can be subtly or unabashedly incorrect. As an engineer you will often be interacting with information that is incomplete, out of date, incorrect, or deliberately misleading or false, and you/your processes/your systems must be robust to that. That's why there are things like failure mode and effect analysis (how likely is it that a problem will occur? how severe are the consequences if it does?). That's why you check- and cross-check info; that's why you do basic calculations as bounds on simulations. That's why you don't blindly trust your finite element model solver when it predicts your part will distort into infinity (the mesh wasn't resolved enough! the node ended up in outer space! it's a computer problem, not a real one). That's why you do things like incoming shipment sampling or verification (the vendor sent a certificate of the material, but they straight-up falsified it and swapped for a cheaper material!) (the vendor fabricated inspection data and said the parts were great but they're not!) (the inspector just isn't very good at his job and measured the parts badly LMAO). Like, you prototype and test your designs and find out what works and what doesn't - what assumptions or approximations weren't valid, what unanticipated phenomena are at hand. Obviously you don't want to waste money on prototypes that catch stupid mistakes: you want to catch those on paper as much as you can before they're in the flesh, yknow. So you do your checks (and this includes, ideally, any info you're putting into your models/designs) and you do your tests, and you check that your shit actually does its job!! like that's... that's what engineering is! and I won't pretend that this process is perfect or that every engineer is a good engineer, but I will say that engineering writ large is built on robustness to imperfect information! And that if your system doesn't catch some subtle AI error,
2) it won't detect subtle human error either! goddamn! Humans are great at making errors!! Humans are making errors all the time! "AI can be subtly wrong" yes! it can! and so can the person who designed a part that some of my parts reference, who didn't update the reference planes and sketches that were specifically for me to base things on, and left things like 6mm off! but I caught the error because I was doing all my cross checks! So can the person who routed some tubing with the valve on the wrong side of a tee and OOPS bypassed the valve completely! So can the person who just entered the wrong number in a spreadsheet! So can the person who suggested some ideas that were just kinda dumb while brainstorming! So can the guy in the meeting today who said something the exact opposite of true because he hadn't paid attention to an update earlier this week! So can the inspection technician who set up the measurement coordinate system slightly wrong so a handful of dimensions are sliiightly off! I bought the wrong barbs earlier this week because I just messed up! And of course, as mentioned above, there are all of the deliberate falsehoods you have to prepare for: fabricated inspection data, incorrect material certs, test results someone cherry picked to save their own skin. Humans are full of mistakes and all systems have to be robust to that too!
3) lol and lmao even.
