The market for programmable devices is huge and continues to grow, driven by the relentless march of technological progress: new applications, better performance and more features. This market is expected to be worth an estimated $15.2 billion by the end of this year, compared to $7.9 billion in 2017. New products are being developed for IoT devices including smart home applications, industrial automation, or Industry 5.0, autonomous vehicles, and generally higher performance embedded systems which require specialized devices. These devices need to be programmed in order to perform the tasks demanded of them, and secured if they are Internet-Connected.
Now, seemingly more than ever, there is a trend towards subcontracting the programming of these devices, propelled by market forces ranging from a volatile chip supply, a constrained labor market, the rising cost of capital investment and difficulty in raising debt.
With these prevailing mega trends at play why is outsourcing your IC Programming a competitive and sustainable option to ensure your projects are rolled out on time and on budget?
Ciara McCarthy: My name is Ciara McCarthy, I’m the Chief Marketing Officer at EPS Global and I’m delighted to chat today to Colin Lynch, who is EPS’s CEO and has a background in electronics engineering.
Colin, can I start by welcoming you to the Critical Lowdown and asking you to give us a bit of background of yourself and how you got to where you are today?
Colin Lynch: Thank you, Ciara. I'm delighted to have the opportunity to discuss this topic. As you mentioned, my background is in electronic engineering. I began my career with Teradata, a company that created the first one-terabyte database, which certainly dates me!
I've been involved in the distribution and programming business since the mid-90s, and I've observed a few trends in the market now that are influencing customers to consider subcontract programming. However, there are also several aspects that have remained relatively constant over the same time.
For instance, despite significant advancements in device size, memory, and speed, the time required for programming has not changed much throughout my time in the industry. This is an important factor for customers to consider when evaluating their options.
Could you give us an overview of the categories of devices that require pre-programming?
Programming is essentially loading software onto an electronic device. People often experience this when they purchase a new gadget and need to connect it to the Internet to download software. This process can be slow, but it's necessary for devices with memory or logic that must be configured before being used in a product.
Memory, particularly non-volatile flash memory, is a key component in programming. While a significant portion of flash memory is used for data storage, there is a substantial market for NOR Flash, which stores operating systems or BIOS code required for devices to function properly when turned on. This code is often pre-programmed in memory.
Microcontrollers, commonly found in appliances like dishwashers and washing machines, also have non-volatile memory and are usually pre-programmed by manufacturers, ensuring that the devices function correctly upon activation.
Logic devices, which are prevalent in numerous applications, are typically pre-programmed as well.
With around 30 billion microcontrollers, 10 - 20 billion programmable flash memory devices, and 5 billion logic devices produced annually, the number of programmable devices is constantly growing. Additionally, the proportion of programmable devices compared to all manufactured devices has been steadily increasing over time.
What are the options for loading the software onto programmable devices?
There's several methods to consider for loading software onto chips.
The first option is that to ask the chip manufacturer to program the devices. This approach has the advantage of receiving devices with the software preloaded. However, chip manufacturers are often reluctant to offer this service - unless you're a large customer with infrequent code changes. They typically set high volume thresholds and require significant order sizes for this service.
It's important to understand that chip companies invest billions of dollars in infrastructure to produce chips. They prefer to monetize their investment directly and avoid adding an extra process cycle for software loading. As a result, this option is usually limited to a small number of sizable customers, so while having the chip manufacturer load the software might be the cheapest and easiest method, it comes with several constraints and is not widely accessible.
What about products that can have their software updated after they’re released?
Funny you should ask, I'm a golfer and last week I bought a new golf watch, a brand new really interesting product from a customer of ours called Shot Scope. I've updated the software twice in the first week based on emails I've had from them. It's not untypical, your software is at its most volatile when your product is new. Quite often, you produce a product and you immediately see things you want to change but you can't change the hardware once it’s been is shipped, however you can significantly enhance the product by changing the software. So, it's not uncommon that the software changes in response to bugs or security threats or feature upgrades. It's a common part of every product cycle.
So it's not ideal to have them pre-programmed very early on the process, so what other options are there?
Programming chips in the semiconductor factory has its place in the market, but it's not for everybody.
Another thing you could do is buy your own machine, which takes your chips and programs them for you. If you have very tiny volumes, you can even do that by hand. Generally speaking, there's limits on what you can do manually, as devices get smaller and more delicate, you're worried about touching them and damaging them, and in any sort of significant volumes, manual programming is not really viable or scalable. Also, there's certain industries like MedTech and Automotive who won't accept any manual handling of devices for very good reasons.
You can buy an automated machine, generally speaking they’re quite expensive, and as you mentioned at the start of the chat, there are a few market forces at play that are causing challenges for people buying machines.
The first thing I see is most of the companies who sell automatic programmers make about half of their money from selling machines, and half of their money from maintaining, servicing, and updating those machines with sockets and software.
It's a fair analog to say that if you are a customer of these companies and you’ve purchased a machine, the total cost of ownership of that machine may end up being twice as much as the price of the machine, ultimately resembling the automatic programmer company's income profile.
When companies are considering investing in a machine it’s important to include the service contracts, true capacity calculations, sockets and algorithm development costs on top of the capital equipment expenditure as part of their TCO analysis.
Additionally, raising capital to buy these machines is very challenging at the moment.
Buying machine is one thing, operating is another. Obviously, you need space, power and light, but you also need some expertise to operate automatic programmers.
There's a lot of engineering every time the parts change; you need to work on new algorithms and sockets; setting up the jobs; and you need to be able to do some level of maintenance associated with the machine.
Quite often as well, a lot of the pricing models associated with these machines assume 24-hour operation. But when you consider staffing an operation like this, there's many places in the world right now where trying to hire experienced engineers on night shifts is next to impossible, it's extremely expensive and fraught with all sorts of risks and challenges.
When you look at staffing these machines, the maximum capacities are always calculated on a 3-shift operation. A 3-shift operation is tough to run in a lot of places in the world. The cost of hiring, training, and retaining staff if expensive and volatile. These problems are greater challenges today for a lot of companies thinking of how they're going to program than they would have been in the past.
Recruitment and sourcing talent is a megatrend businesses are battling across the board. It's not just trying to recruit mechatronics engineers to maintain these machines, it’s basic shift staffing challenges.
Across the world, people are struggling to find people who will work on-site from 9 to 5! Getting people to work night shifts is now is a serious challenge. At EPS Global we don't operate like that, generally speaking we don't have a 3-shift operation anywhere in the world. As we design and manufacture our own machines, we tend to overpopulate our service centers with equipment and operate 2 shifts in centers where we're particularly busy. But I think if you're trying to staff a production environment now before 7am or after 8pm, you're going to have serious staffing and quality challenges.
If I buy a machine specific to a job or my investment decisions are based on upcoming projects and my project or device mix changes, can I still operate the same machine? How does that work?
You can get lucky or not, it just depends on what the mix is and what it changes to. There's a lot of good third-party manufacturers and quality equipment out there. It does often work with many different device classes, but not always. Sometimes you could have something that's working very well for microcontrollers, and suddenly the product mix switches and you may have higher density memory arrays that can significantly slow down your programming time.
Again, the machines are usually upgradable so you can update them. But that's expensive, and as I've said – the cost of updates is where this concept of spending double the cost of the machine over its life comes from. You can maintain and update the machines, but it takes engineering time to specify those changes and it costs money to upgrade them.
I think the bigger challenge regarding machine ownership is to do with the volatility of chip supply. There's been a lot of volatility in supply chains in all markets, and there's still a significant amount of shortages in the automotive space.
When you're buying a machine, you do a capacity plan and you will probably over-provide, for example if you're trying to make a million units a year and you buy a machine that's capable of 1.2 or 1.5 million, you probably feel that you're reasonably well covered. If your deliveries come in at a hundred thousand a month, that's going to work fine, but if you don't get any deliveries for 3 months, and then you get 300,000 at once, your machine with a 1.5m production capacity is going to take a couple of weeks or months getting out from behind that backlog. During which time now you have the product, but you can't use it because it's not coded, and so, volatility in supply can really play havoc with your capacity planning.
This has been a feature of our lives in the electronics industry for the last 4 years, and in certain device classes - particularly in the automotive space - it could be with us for some time to come. So again, that's one of the other things that I often think about when I look at the real experience of buying a machine versus how it looks sometimes on paper.
At EPS, what we're doing is making and staffing our own machines and doing the programming for you. So we are programming with the same type of equipment (that is designed and made by ourselves), and we're providing a similar service to what you would do yourself if you bought a machine.
What is Programming at In-Circuit Test (ICT)?
In every manufacturing line, when you assemble a circuit board, it requires testing. The testing usually happens at the end of the line when the product is fully assembled and is called ICT. It's bespoke, designed to test a particular product. During ICT, you can take the time to load software onto your flash memory, logic or microcontrollers which is a very logistically simple way of doing it. Your product is there, it's in your factory, and the chip is already on the board. It's always the last stage in the production before you ship out, so you're able to load up the very latest version of the software and the product is often shipped the next day. There’s little logistical friction associated with that, and a lot of devices are manufactured this way.
The challenge with that approach really comes down to the programming time or how difficult the programming is to do.
For example, if you have a tiny 8-bit micro that has a one second programming time, and you've a product that has a 20 second test time, you won’t see much delay in programming that type of a chip in test. It's not going to make an appreciable difference and won’t impact your productivity too much.
If you've got a 256Mb flash array that takes much longer to program, you’ve now exponentially increased your test time, suddenly you're making only one device every time you would have made several devices. It’s a real challenge. The utility of the entire line can often be gated by the time it takes to test, because it’s the lengthiest process on the line. A production line that makes a complex circuit board is a multi-million dollar investment, so you do not want to increase test time significantly, adding delays to the line which ultimately cost money.
What is usually done is putting testers in parallel so that there's enough testers to keep up with the line. But this requires more space and it can make your production flow a bit clunky. Generally speaking, the testers used to test circuit boards are expensive, much more than the programming machines you use to program a chip. Your cost of capital jumps each time you add a tester to the line.
Generally speaking, Embedded Engineers who are designing microcontrollers will try to design the programming time to be small enough so that In-Circuit Test will be an option and may even be a really good fit. But as I mentioned earlier, there's always more memory. Applications will always get bigger.
For example, when you add IoT or Bluetooth connectivity to a microcontroller which might be 20 - 40Kb of code, every one of these applications adds code on the previous revision of the chip, which can double the programming time. Moore's Law obviously applies to programmable devices the same as anything else, but as the I/O to the chips has gotten faster, the chips themselves have gotten bigger, which combines to keep the programming time the same.
Even though the speed at which you can program data has increased over time, the amount of data and complexity of the application you need to program because of the addition of new features has increased proportionally. In my experience, the programming time hasn't changed a lot over my 20/30 years in the business, but we’re now programming a lot more stuff in the same time.
How can you improve the programming time of the 256Mb flash array I mentioned earlier? If you outsource the programming of your devices to EPS Global, we charge a per unit price. We program, mark the devices, and then we deliver them to you in tape and reel so they are ready for the SMT process on the line. By the time your device comes to ICT, it has the software already loaded.
Your test time is immediately back to 20 seconds in our flash array example. Yes you have increased the price of the device because of the additional programming charge, but this is quite competitive relative to the cost to your business if your line’s productivity has been cut by a half or a quarter due to programming during test.
To illustrate: if you program at test you might save yourself 10 to 15 cents of a programming charge, but surface mount lines can be a $10 million investment, and that’s a lot of capital tied up waiting on the ICT process to finish. However, if your parts arrive to the line pre-programmed, theoretically you could produce 2,500 pieces a day instead of 2,000 which increases your efficiency and revenue. The per unit programming charge on the device starts to look like a significant cost saving to your business.
There's lots of examples where ICT is a really good solution for the customer, but also there's many situations where depending on the class of the device, any new features introduced can cause the programming or test time to increase significantly which moves the dial on the decision to outsource very quickly.
Secure Programming, or Secure Provisioning, is the first feature that comes to mind because many devices now are going into connected products. An unsecured Internet-connected product is vulnerable to hacking, cloning, counterfeiting and data interception which has brand reputation, IP protection and end user damages to consider. Already enacted and upcoming legislation enforces penalties on the C-Suite executives of organizations who do not secure their connected products which range from steep fines to jail time. We’ve a podcast and blog on this topic here.
Consider your programming time is 10 seconds on a small micro, but now you've added a step to reach out to the cloud, validate or authenticate that device, and download a key. The actual code base takes 10 seconds, but the call home to get the keys and certificates could be an additional 20 seconds. Additionally, your existing ICT tester is not equipped to manage encryption and authentication, you must add a Hardware Security Module (HSM). If you’ve more than one tester, you'll have to duplicate that hardware setup on all of your testers.
For a subcontractor like EPS Global who specializes in Secure Programming & Provisioning and has a global network of secure service centers and machines equipped with HSMs, adding 20 seconds to the programming time is not significant. It doesn’t make a huge difference to our pricing, but adding delays to your line if you decide to add secure programming at ICT can really move the economics in how you measure the productivity of your factory.
If I look out 12 months to 4 years, security is going to be mandated in the industry in a whole series of consumer devices where it hasn't been before. Programming times will increase in order to enable security. A lot of companies that previously used ICT will be looking for solutions from subcontractors. That's one of the big markets I see.
Absolutely. With high volume production of connected devices, the demand will be massive.
You've talked about the available options for loading the software onto devices from pre-programming the devices in the semiconductor factory to buying a programming machine to programming at In-Circuit Test.
You've also discussed prevailing market factors including labor shortages for staffing your line, the volatility of chip supply, and the difficulty in raising debt to buy capital equipment. These seem to be the key factors that are impacting the decisions to outsource chip programming to a subcontractor like EPS Global.
What’s the advantage of using a third party to program software onto electronic devices?
There's lots of other companies who are in the subcontract programming game. EPS Global the most pervasive - we're the most global; we're the only programming house offering the breadth of services that we do, and the reason why our global footprint looks like it does is because of the growth in subcontracting in the electronics manufacturing industry due to the market forces you’ve outlined.
If you partner with EPS Global you don't have to buy any capital equipment, your risk is substantially reduced. You don't have to hire people. Your cycle and test times are all massively improved. Also there's a strong probability that you will experience a quality improvement because our quality management systems are quite specialized, we're better able to adapt on code control; we've biometric ID access to machines, we’ve got traceability on the devices, the machines they were programmed on and the machine operators and we retain this information for 7 years should you require it. Our processes are honed to cater for the Tier 1 Automotive, OEM and Contract Manufacturer customers we have and to pass their stringent audits.
For balance, we need to look at the disadvantages of outsourcing your programming to a subcontractor. Generally speaking there's some logistical friction because you have to send your devices to another location to get them programmed and then returned to you so there’s purchase orders and transport involved. There's also a charge.
A significant portion of our strategic planning is spent in figuring out how to solve the logistical and pricing challenges facing our partners, which are the only two disadvantages of using a third-party service, and this is why we have structured our business the way we have.
Manual handlers cost from $5,000 to $20,000 from various manufacturers around the world. Automated handlers can start at $300,000 and run on up, depending on the features you need the cost increases. For a company like ours with almost 50% of our business in Automotive, we need every bell and whistle that a programming machine can offer - tray stackers and tape out, laser markers, 3D co-planarity checking to make sure the device is still planar after it's been in a programming socket. When you add in these types of features, programming machines run north of half a million dollars, even for ones with relatively modest performance.
But… we integrate these features into our machines ourselves. We’re very lean, we have a highly expert team of Mechatronics Engineers and Software Engineers in Brno who make these automatic handlers in-house, and the handlers is where most of the cost lies. Our lower capital costs allows us to pass the value onto our customers.
The combination of slick logistics due to our close proximity, and competitive pricing is our best effort at minimizing the two disadvantages of using a subcontract partner.
What markets does EPS Global address?
That's an interesting question because I believe that the two biggest types of customers we address today are going to be quite different from what we're going to be doing in 3 year’s time. Let me explain what I mean by that.
That's our current customer profile, but what I talked about earlier in respect of security is going to change that paradigm slightly. Today, there's a lot of consumer micros that are programmed at ICT that are suddenly about to have to consider security: "We need a completely new hardware setup and we need to double our programming time, and is our productivity dependent on an Internet connection to a server somewhere?".
This example is another piece of the security puzzle. Getting and providing your keys online securely. What if your Internet connection suddenly goes down? Your production line stops. These issues are going to change the profile of the types of customers we're going to look at, and significantly.
I think security, and provisioning security, is going to be a big challenge for the large chip companies. We are partnering with a number of the biggest companies in the microcontroller business, and there will be consumer OEMs we'll work with in the future to provide security services who in the past wouldn't have looked at subcontract programming partners like us. I think the paradigm will change, as soon as the next 2-3 years.
We've discussed the market forces that are currently at play in the industry. Is there any evidence that there is more of a trend towards subcontracting programming more than before?
How are these macro trends impacting EPS Global's business?
We're experiencing a very accelerated rate of growth at the moment, we may grow as much as 30% in our services business this year. We’re assessing what the business is going to look like beyond next year which, when you look at a shift in something that’s typically been done in one way, like programming at ICT for example, may now move to an outsourced function, this may actually change the addressable market.
The market for services is growing and we have a lot of customers reaching out to us looking for various types of component configuration support, not just programming, which is changing the attribution of the service mix in our business. The growth rates of 30% we're seeing at the moment may sustain over a relatively short period of 1 - 2 years, but I think it might run beyond that, we have very aggressive goals.
Businesses who are building connected or IOT products have to answer questions on how they’re going to work through their security compliance commitments. There's certainly a few flavors of how they might address that, and that will result in a significant increase in the total available market for us as we are ready to service security requirements for customers. We're reflecting our expectations in that market with investments we've been making in our capability to build equipment at the moment. We're heavily invested already, and we're doubling down on that investment in terms of increasing our capacity for machine manufacturing.
Having oversight on our marketing strategy and activities, I can see the positive effect nearshoring programs and the need for securing connected products are having on the leads coming into us. Really there’s several megatrends at play worldwide that we're seeing the benefits of that might indicate sustainable growth.
The economics of why you would use a contract partner for programming are moved by smaller trends than just what's going on in security, it's more about simply outsourcing services. With nearshoring policies, we’ve seen the contract partners we work with in Eastern Europe and Mexico getting extremely busy recently. There's a lot of business moving to those companies and they’re working really hard to staff and stage the influx of business.
The ability to have partners like EPS Global who can take some activities like programming or taping and reeling out of their portfolio of problems, and work with them on solutions is certainly something they're keen to take up. There's some parts of the world that are growing very dramatically due to nearshoring, but where we haven't seen that growth is in China. I think the security piece is going to make a significant difference. I'd be very optimistic about some aspects of our Chinese business over the next 3 years for that specific reason.
Generally speaking - changes are positive, good problems in terms of how to cope with growth at a fairly accelerated level, and our R&D group and our development group are working hard and all of our service centers are extremely busy at the moment.
Normally at the end of a podcast I ask What’s Next?!, but I think you've covered it in the security market discussion and other areas our growth is coming from. But if there's anything else you'd like to add, please go for it.
Honestly I think they're the most important things. It was great, I really appreciate the opportunity to chat through some of these macro industry trends, because it's not often you step back and think “there's some changes going on here that seems a little bit more than the sort of steady growth that happens typically year on year”.
I definitely think how people are working now is making a big change in which companies are interested in subcontracting or not. The labor constraints and talent sourcing issue is going to be huge for our business. You look at the difficulty of getting people to physically work in an office 5 days a week, that's going to become a huge challenge for companies in the production manufacturing sector. Our approach to that is easing the requirements for additional staffing for the customers we work with by taking on their component programming and configuration, some of the more labor-intensive activity.
And then, the other big megatrend is security. The PSTI bill (Product Security and Telecommunications Infrastructure Act 2022) became law in the UK on April 17th 2023, and there’s legislation in the EU and the US coming in the next 18 months. In the first 3 months of 2023 we’ve seen more leads than we saw in all of 2022 in our security business. It wouldn't surprise me if that growth continues.
The future sounds bright! Thanks for joining us today, Colin.
I appreciate the chat. Thanks very much, Ciara.