Diagnosing a no boot / POST remotely

[quote]multiport pci serial card.[/quote]

What is plugged into this card?

Is it possible you're getting some kind of ground loop through a serial cable?

If it's happening across multiple machines and multiple PCI cards, I'd take a long, hard look at what they're plugging in to that PCI card.

Hmm. First, I'm a little fuzzy on the symptoms. To clear that up:

You say "that operate remotely", which sounds like it's a headless server, but you also say "there is no issues with the monitor". Do you mean that they attached a monitor to a video output and that they could see video display?

You have:

[quote]We’re unable to access the bios when the computer stops working.[/quote]

Like, is this via a video output, serial port, or some kind of dedicated hardware management system?

As to troubleshooting, when you say "no packets", and since it sounds like you have access to and familiarity with the router, you mean not even stuff like ARP? Like, it's not "I'm not pinging anything", but "no Ethernet frames have reached the router on that port?"

I'm...a little confused about you saying that the NIC is negotiating speed, because I'd have thought that on typical systems, a NIC wouldn't be doing that until the OS is up and tells the NIC to become active. But it sounds from your systems like whatever is happening is happening pretty early in the boot process, before the OS or even bootloader is doing anything. Do you have the BIOS set up to make use of the NIC in some way, like using DHCP or BootP or something to do network booting?

EDIT: Or wake-on-LAN?

EDIT2: Because if you do, and don't actually need wake-on-LAN or network booting functionality, I'd think that I'd try turning it off to see whether the issue might vanish.

[quote]This particular port drives a DC motor. Power is injected to drive the motor.[/quote]

On one hand, that does sound suspicious. It's an external factor and it's unusual.

But on the other hand, you said that having it not plugged in doesn't consistently see the thing working, which doesn't really mesh with what I'd expect if it's the cause.

You can get optical isolators for serial ports. Those basically run the signal into an LED and then to another LED, which keeps two separate isolated electrical system; those will eliminate ground loops. I haven't looked into them for serial ports, but I was looking at hooking up a breadboard to USB at one point, and there it seems to be kinda a best practice for USB device development work, to keep any mistakes from damaging a connected computer system.

https://www.amazon.com/rs232-isolator/s?k=rs232+isolator

In your case, it sounds like you are also using the serial port as a power supply, so you might want something that can provide external power:

https://www.amazon.com/External-powered-Repeater-Mini-size-PhotoElectric-Full-line/dp/B00GI9GRMC

It might also be possible to use an isolation transformer, which would be simpler and possibly cheaper. That would, I believe, provide ground isolation without providing protection against more exotic things, like a short in your external device frying a serial controller. I've used isolation transformers for coax TV to avoid ground loops, so I imagine that it must be possible to have them handle serial port speeds. But when I search for "RS-232 isolation", everything I see seems to be opto-electric.

If you can change the serial port being used, maybe also use a different serial port interface, like a USB serial interface.

Power requirements are different around the world. It very well could be relevant. You mentioned a DC motor, which has me wondering if it's in an industrial setting, and if that setting might have some unusual variety of 3-phase power rather than split-phase 120/240 typical of North America.

Should have been more clear about the remote part. The systems operate remotely from me, I can access them via the internet. The users need to use the screen to operate it. This is just a windows 11 computer after all.

As for the router information, thats part of the reason I know there is power to the entire thing. I can access the router page and see that the link speed was negotiated correctly, at 1gb per second. If they unplug the ethernet, the port reads disconnected.

But 0 packets are getting sent from the computer. I believe those are ARP, but the router page doesnt define it. Just has a table with packets in and packets out. Packets going in will usually have a couple from the router. Always 0 with packets coming out. Theres actually a ping function built into the router, and that doesnt respond at all.

BIOS is not setup during boot to use any sort of networking. No PXE boot or anything like that. On this particular motherboard, you have to enable the network stack, so I dont think its that. System is setup to always turn on when power is applied to bring all the other components online together.

Anyway, thanks for taking time out of your day to respond!

Now that is an interesting idea. I cant go into to much detail about the exact components but this particular port drives a DC motor. Power is injected to drive the motor.

Tbh, im not very familiar with ground loops. Have you had experiences with ground loops before?

I appreciate the quick response as well!

Ooooo now this is something!! Ive always had a hunch that the way power is being injected into the motor was causing the issue. The serial device is connected by a DB9 port. The RS485 signal A, B and ground then gets accompanied by power to drive the motor. The serial port from the computer goes into this power injecting box and then the box goes to the moto and controllerr.

The grounds should be connected correctly. But now youve got me thinking about these boxes. Modifications have been .are to them overtime, so its totally plausible that something in there is causing a surge to get back into the computer.

Thanks for the response!

Oh wow this is a fantastic write up! I really appreciate the effort you put into this. You've certainly given me lots to ponder over. I wonder if there is some credence to the humidity issue. Id have to do some digging on when the issue occured and what the conditions were like but some of the locations do get humid.

“a couple of dedicated machines”

Is this a new installation? Has this process control system ever been in a state that did not produce the reported discrepancy? What changed and when?

Did you virtualize the site’s legacy system?

It seems like the problematic variable is likely to be external to the PCs themselves. My inclination would be to have the end users disconnect everything but the monitor to see if the systems POST, and then reintroduce peripherals one by one.

The ticket said "Every time I sit down at my desk, the screen turns off."
So I went on site and told the user to demonstrate the issue.
She sat down on her chair and the screen shut off, then turned on again a couple seconds later.
It was a combination of static electricity and the telescoping chair stem, which induced a magnetic field that disturbed the monitor.

Power to all components is provided through a dedicated power supply. If the computer was on, I would be able to get direct readout of voltage and amperage. As for the serial ports, some devices use RS232 and some use RS485.

The ground loop certainly seems to be the most plausible cause. And when its in the shop I've got exact replicas of the system. I just cant seem to replicate it no matter how hard I try.

For context, Ive stuck a system in a fridge to see if possibly cold temperatures correlate and was still unsuccessful. At this point I wonder if there is interference coming from an outside source and thats why I cant pin it down.

You've given me a lot to think about! I greatly appreciate the responses.