Asianometry - 2024-06-30
Links: - The Asianometry Newsletter: https://www.asianometry.com - Patreon: https://www.patreon.com/Asianometry - Threads: https://www.threads.net/@asianometry - Twitter: https://twitter.com/asianometry
I once was a field engineer for ASML/Zeiss and regularly replaced lenses in those machines.
It surely was an interesting time, but it’s only for single people who are willing to travel on very short notice.
The good thing are luxury hotels and always a few days paid leave in the country.
I personally enjoyed South Korea the most, I was there stationed for 6 months and lived in an amazing hotel in the middle of Gangnam.
Now that I have a wife and a house I switched to a job that’s 100% remote.
It was a great time and the team at Zeiss was absolutely incredible and gave us the best training imaginable.
I will never forget my first time in a fab, deassembling that machine.
It felt so surreal!
What kind of remote job do you do now?
What are you doing now?
What sort of qualifications you need to pursue a career like that?
Did you ever fear "OMG what if I can't put it back together again?"
Cap
“Right so the computer that makes computers for fixing other computers is broken”
😮😮😅
“But Have you tried turning it off and on again?”
Yes, very much broken like your cat that sleeps in its food bowl, rather than the bathroom sink.
don't forget that no machine could get these things up and running. the machines can do the impossible, but only with human help
The ASML field engineer only makes between 90 and 110 grand a year and has to travel all the time, often on short notice (but at least the long flights are business class). The job is really tough on your body, with long hours of standing and moving heavy equipment. And you have to do it all in a clean room environment. But on the plus side, you get to work with the latest and greatest technology.
i can understand how they accept the job, many times i've seen that being on the cutting edge or in a job position many would like, turns out this way.
Game devs, tho not so cutting edge, have the same overwork/underpay issue
That’s horrible pay! Equipment engineers around here make up to twice that from what I’ve seen.
Now adjust for a country with policies that keep housing affordable.
@@myne00 ahahah holy shit 90 -110 grand is what they made 10 years ago, their wage hasn't changed at all, back then it was a REALLY good job eh
That's the same pay I had as a former field service engineer working on digital cutters made by the Swiss, mainly in the printing and packaging industry. It's not enough money and yes, I am totally ruined health wise because of it. Started when I was 23 and barely made it intermittently working to my 30s, late 30s now with disability due to many unfortunate events that were all in some way stemmed from my job.
"A wiki of sad times" is such a fantastic quote.
Agreed.
I liked the quote but couldn't understand it fully. Can you elaborate? 😊
@@RJ.Mangal It's a knowledge base filled with issues, so it's a wikipedia page, but filled with only problems. Hence, wiki of sad times.
Yeah homie is witty & funny in these masterclasses =P
I used to work for Hewlett-Packard analytical instruments and one of our customers had to pay a penalty of 25 thousand dollars per day if the report was not filed.
We had a lot of pressure to get the machine back in line. I can only imagine the pressure the engineers at ASML suffer.
My work, we have a removed tertiary involvement in wafer production where we have different FSE's on site for our "tools"... there is pressure on them to get stuff back up. (I'm applying it) But they do what they can, get what parts they can when they can, then go home. Me, I oversee the entire building... if we have a total down, I don't have the buffer of being just a paid vendor. The FSE's don't feel the $250K/day loss like I do. Had a few of these days so far this year, let's just say I no longer have a functioning digestive track. While I'd never want to be one of these ASML FSE's, I'd really never want to be the fab manager in charge of that ASML tool... hats off to those guys.
That's why HP charged 80 dollars an hour for our time.
45 or so years ago.
I worked customer support for warehousing equipment (big sorters, conveyors, etc. for some of the biggest distributon centers you can think of...) and they sure hate downtime. Was on a 14hr long call with one place once. We had to get support engineers from another company on the line, and there were VP-type folks from the company experiencing the downtime on the call. The support engineer said he'd have to escalate the issue to the other engineers on his team, and when asked how long that'd take, said "they guarantee a response within 5 days". I could practically hear the customer's executives' heads almost explode.
Well, that's what support contracts are for
hope u are doing better, noticed you were down last video, dont know what happened (if anything) but just wanted to let you know my (and many many more people) appreciation for your superb content, subtle humor and excellent information! :p
I think we all noticed and worried. He does sound more like his usual self.
I can't even watch the video right now because I have work tomorrow but I came here to say the same thing
This man is a treasure
I hope he is ok
I think he might of been just sick or tired
I am feeling a lot better. Yeah I’ve been a little sad lately. Thank you for asking
@@Asianometry
It's ok to be sad from time to time. Thank you for sharing.
I hope your tomorrow is a wonderful day. You make the day a little more bearable for so many people. And you fill the world with wonder.
Thank you.
This hits home hard as a former field service engineer. I worked pretty much exclusively on a Swiss made digital cutter line and it was very much like this too, these machine were always the bottleneck and key equipment in all the businesses that had them, so when they went down it was most often a big deal for the customer, every minute was costing them and their own customer's money.
More complicated than a Stage1 Turbo 6 I'm sure!
I bet you when it goes down there's always that one guy that screams "fuuuuuuccck."
It’s quietly muttered by many, it’s bad fab managers that yell it out.
It’s like a power flick in a fab, as the turbo pumps slam back on shaking the building you hear everyone involved in the recovery sighing f::…
always a beaker 😂
Lol
as someone who suffers from an incurable, painful disease, with no effective treatment, I yell swears all the time (not directed at individuals) . there have been studies (including one released from Harvard University) showing that swearing for pain relief is as effective as opioids.
Yea you're mom
A Scanner down creates a lot of havoc on the track side as well. It's a rare opportunity to get some work done. Pretty crazy how much info you got in here. It'd be interesting to see what you could say about track dispensing purity and health
Heh I did my undergrad dissertation in Comp Sci on simulating and correcting optical aberrations in human eyes. Made some nice wavefront visualisations based on Zernike polynomials.
I would guess 99% of people have no idea how complex and coordinated (between many companies and countries) making chips is. Thank you for making some of us a bit more knowledgeable. Literally, a bit.
Sadly though the high iq populations genetically capable of doing this complex endevaour are dying out due to their low TFR's
The overwhelming complexity and size of the body of knowledge always makes me think how fragile our modern world is. These chips are, more than anything, what makes our way of life possible. And the amount of expertise necessary to produce them is mind boggling. Losing just a small percent of these experts would grind it to a halt.
@@Norsilca to me, and perhaps Asianomitry might allow, it’s really beyond something that can be taught, or learned. It’s art.
The other 1% make chips
99 is a really big underestimation in my view. I would guess one in few thousands has a surface level understanding of most of the main steps in production.
I am not in that category of people. And I study electronics, something I suspect 99% of people don’t do.
Thanks for the ASML video once more. I am a mechatronic engineer and work for ASML for 30 years, so I'd like to follow what is published about ASML. But I was most struck by you mentioning the book of Marc Hijink...who was raised in the same village I am and who wrote an excellent book about ASML. Together with the book by René Raaijmakers, it's the most informative book I ever read about ASML. His way of writing is fascinating imho.
And your pronunciation of his name was 100% spot on!
Although drilling an oil well may not sound particularly high tech, it is. A dynamically positioned drill ship may be drilling in water 1500 meters deep with a hole 1500 meters or more below the sea floor. All this relies on many things working properly. When somethings breaks, like the systems that keep the ship on station, all hell breaks loose. It's a million dollars or more per day to operate the ship, and the ship may be 200 miles from shore. The technicians on board are the best of the best. They have to be.
Bruce Willis used to do that, but they needed him in space and he had to save his daughter's boyfriend and he was the first person to die in space.
We dont care about that.
Those dynamic positioning systems are so critical. There was a diving support vessel years ago that got caught in a storm and its DP system failed at the worst moment, began drifting while divers below were still tethered to the vessel. Led to one of them having their gas and hot water lines severed leaving him thousands of feet alone at the bottom of the sea with no air. Crazy story, he survived and was rescued for the record, nobody knows exactly how he survived so long without air. Name is Chris Lemons for anyone who wants to check the story out.
Who said drilling for oil isn’t high tech? They have no clue! Oil companies are tech companies. Tech doesn’t just mean computers - oil companies deal in the physical world - mechanical tech.
I dunno anything except that 300 wafers/hour is astonishing, incredible, fantastic and amazing and all the other superlatives....
To me it appears that these machines are like the absolute peak of terrestrial technology...
As a principal data engineer working on the flow of data coming from these machines, I can say I learned a thing or two from your presentation. Great video and keep them coming.
Ex-ASML employee here, actually the most pressure came from tsmc, once upon a time there is one machine with down time long as 3 days lol, no one knows how
haha is that the record?? only three days? seems like a minor/moderate outage (I mean obviously expensive but three days is pretty good imo)
@@nixic_ unexpected down time for three days and no one knows how(on CS team, in production), the longest down time I have experienced is in installation phase, unexpected down for about 3 weeks, it was pure pain
@@lunamiya1689 the longest downtime that I have ever experienced when I was still working for the chip industry was that little fire that made DDR RAM chips quadruple in price for some time..
You already worked back then?
When you make die using a lens that's slightly out of focus you get fuzzy logic.
0/10 joke right here hahahaha
@@davidanalyst671 Nah, thats a great joke.
@@davidanalyst671 collapsed right into a 0
😵💫 Fuzzy logic! Fuzzy logic! 😵💫
Good one haha
My cousin works at the ford plant and at $40,000/min for downtime, they have a helicopter ready to fly to another plant to get parts if they need it.
Ford = Für Opa reicht das
That’s German: Good enough for grandpa ;)
It ain't fun being a lead Engineer on support where your task is not complete unless it is 100% rectified, tested, commissioned and signed! Sure the money is great... but the pressure makes one feel life is getting shorter by the second. Glad I'm retired! LOL!
The money is great?!
@@yaboifet9058 IT is the irony of it all, that in the end it ain't worth it. Life is to be lived and money has nothing to do with it!
How much money a hour?
@@yaboifet9058 110k per year in a country with cheap groseries and almost free healthcare.
i work as a field service engineer and this video has a scary amount of information i wouldnt expect people to know
But do you feel that your work is finallyy being recognised and appreciated ?
Oh my nightmares are back. Get all the way into the Fab and realise you forget something back at the office. Three hours later you back through and the customer is screaming.
Get in the fab , gown , speak to client people solve problem, talk to fab people leave fab , degown go back to desk have an call/email waiting saying they need you back in the fab.
0:27 when you learn an hour is 100 minutes long.
Each hour they have 40 minutes of bonus losses.
Do you even math?? 1.800 x 60 = 108.000, 1.800 x 100 = 180.000 😂😂
@@OorZakO The videos can be updated after posting now fyi.
Lol
At the computer chip company I used to work at there were people I regarded as 'million dollar engineers'. They were so exceptional that an entire project/product depended on their ability to debug critical issues. But they were never paid a million dollars.
Imagine something that makes not a single machine, but an entire cleanroom go down. Like a malfunction in the air ventilation system, or a fire alarm that gets triggered.
Being a simple maintenance guy at a fab must be insane.
I remember sitting next to an ASML employee and fellow Dutchman once on a flight from LA to Amsterdam. And even though he was candid about his workplace and I have a mechanical engineering background and an interest in computer hardware in general I didn't get much wiser than that he did something with software for his employer. This was not too long after the IP theft case in San Diego became news, so I could imagine that they were extra careful already back then.
"Wafers per hour" as the one metric is mentioned at 02:24. It was during EUV development the goal to finally get there, starting from "hours per wafer". So a whopping 10 years into it the ASML teams finally made it.
All the technicians and engineers keeping our world running deserve maximum respect. Being up to the tsak and bearing all the responsibility plus hard work on their shoulders is mind bugling. You couldn't pay me enough to be in that position.
The first rule of ASML club… you don’t talk about ASML club.
If this is your first wafer… you have to fab.
If the foil sticker peels off the welle lens, the fab stops and you have an unscheduled down.
@12:50 "Like a traffic jam on the 405" - that's the 5 freeway right at my exit in Tustin California 🙂
Your channel is a gold mine of knowledge about CPU manufacturing. Keep it up. Thank you.
It takes lots of livers. A lot of livers.
?
Production go Liver me Timbers.
@@JosGeerink Being constantly exposed to mental stress, take its toll on the liver. You can end up with cirrhosis that'll permanently damage the liver and even cause death if not diagnosed on time. Stress and alcohol abuse do the same damage to the liver.
@@irasthewarrior Did you ever think of combining these two?
money pouring down the drain and you get that customer support guy "welll in 48hours we can escalate it..." 😱🤯🤯🤯
That’s when you get your bosses boss to call their boss 😂
We only talk to line up our ideas and strategies then go to lunch 😂
You nailed Hijink, and another great upload. Thanks!
In my younger days, I worked on CD measurement SEMs. The drag with those is there was usually just one per product line, so when it went down, the whole line was down. It felt like all eyes were upon me as soon as I entered the fab. I knew if the problem was in the electron gun, it would take an overnight pump down to reestablish vacuum. I used to think those were fairly complicated systems, but compared to modern litho, they seem simple.
I'm a cnc machinist studying EE and was hoping to get a fab job. This is super interesting to me, you don't see very much on day to day operations in the fabs. Thanks!
The field engineers for these machines must be being paid in gold.
The sad thing is that they aren't. They are paid decently for hardware engineers, but nothing close to software.
70k/year state side with a bachelor, endless unpaid overtime
@@dougdimmadimsdale9571 Yikes, I hope not in a high cost-of-living state. And unpaid overtime!? I understand for a salaried office-worker but how is that even a thing for an FSE?
@@dougdimmadimsdale9571 Is this current? Because I know that last year they offered 80-85K for new grad production engineers in the northeast US. It's hard to believe that field service would be less than that.
@@apexaviourhigh cost and high taxes. but of course biden claims 6figs without a bachelor, yeah right maybe with 15years of experience
48 hours?? Imagine having to lose $5,000,000 to downtime before you can be escalated to level 2 support
You're playing semantic games here.
This level 1 support is any other company sending in The Wolf.
This level 2 support involves looping in S. R. Hadden from his penthouse suite on the Mir space station.
This level 3 support involves the Space Guild executing space jumps to fetch a dream team from Zarkon 6 from clear across the other side of the galaxy.
The correct semantic frame is this one: Imagine who is on the line when you have already lost $5,000,000 to an issue that not even The Wolf working consecutive twenty-hour shifts managed to mitigate or resolve.
Extremely illuminating video, thank you! Talk about metrology's role and how important this is for the process. To add to the confusion when confronted with the relevant details, the topography deviation chart at 01:17 is upside down and mirrored, too, making it really hard to understand that effing <0.3µm x 10⁵ is the range we're talking about . ;-)
With such costs per hour of downtime, I kind of would've expected a team being at the ready at all times to jump into "GO! GO! GO!" mode, working at pit stop speeds to change lens elements or whatever.
Your videos and information are truly excellent, and I have no idea how you'd do this in your spare time whilst (presumably) holding down your main job.
Awesome stuff!
12:52 Traffic Jam on southbound 405 at Tustin 😂 Nice find! 😂
Mostly watch these videos since I got interested, when I was doing research prior to buying the ASML stock. It has turned out to be one of my best investments, and the lithography process is pretty fascinating. I have a history in the medical industry, building machines that sort proteins. The sorting was done using monochromatic UV spectroscopy.
When an ASML machine goes down, I get paid well. I word in repair. Motion Control repair is a Great STEM field for those who are able to learn and be effective in troubleshooting.
The process of overlay is referred to as Registration, where one layer is registered to the prior layers.
Great video. This push for super efficiency and performance to recoup costs and maximise ROI reminds me of London Heathrow airport. A plane full of passengers takes off or lands every 30 seconds... until something happens (too much wind requiring greater safety distances, a single plane aborting landing and having to go round to rejoin the queue amongst many others). This single exception ricochets through all the flight schedules across European airports as the exception causes a slight delay which causes further delays through the day until the airports close at night and some attempt at recovery can be made overnight. As a seasoned flight passenger, the learned lesson of 'stay at an airport hotel overnight and get the first flight out' is a sure-fire success strategy! Perhaps Asianometry could make a video about the tuning of airport and airspace efficiencies?
My old company would have been bankrupt if they lost this amount of money per hour. Why? They would waste a whole day trying to find someone to blame first, another day to find someone who has free time to go look at the problem, another day to fix the problem, complain why they lost productivity and finally complain about the money lost.
What strikes me most is how many different frikkin parallels have been drawn with Interstellar!
and how many people think everyone sseen it when its locked to a network.
@@luminousfractal420 Bet less than 35% of the US population missed Interstellar.... It's like mentioning the Titanic or something, we ALL watched it lol
How so?
What? The machine god in your lithographorium is angry? Don't worry. The Adeptus Mechanicus will send its most learned techpriest and the Departmento Munitorum will ship the neccessary parts. The emperor protects!
Now just wait till you hear about wafer scrap. Now that's the real kicker.
Broke a boat load of 4" wafers at NMOS 4, Ed Bluestein.... too much coffee! Luckily at the 03 level, just added poly contacts.... not as bad as 07 after metallization. Used to run a Leitz wafer microscope, inspect 1000 wafers a night... I think they put meth in that coffee.
@stevestarcke - 2024-06-30
I was engineer on a line that cost a million dollars a day to be down. What a job that was. Talk about motivation to keep ahead of problems.
@runthejules91 - 2024-07-01
that is crazy.
@mefobills279 - 2024-07-01
I listened to management kvetch about how the technicians were overpaid. This in a factory that profits were in the millions of dollars per day. Get your MBA and get brainwashed, then export the jobs to China.
@craigmcmeechan5899 - 2024-07-01
I've worked as a contractor at a chemicals plant doing unrelated maintenance. Day of the maintenance a Pipe needs closed off and the plant looses around £100000 each hour. All maintenance work for that sector isxplanned and choreographed to perfection, every risk foreseeable is planned for and expensive (almost never used) redundancies/workarounds paid for
@letsburn00 - 2024-07-01
I've been at a place like that too. $7m/day per train and we had 2 trains..when there was a shutdown (called a trip) it would take 24-36 hours to get back to full production.
@tonysheerness2427 - 2024-07-01
You want to see the panic when credit card clearing ,machines go down.