stas' weblog

What would you do if your D-Link (TrendNet, Allied Telesyn or other crappy manufacturer) broke after the warranty period is exhausted?

Recently, exactly that happened to me: our DGS-1248T manageable switch, which we used to connect non-critical servers to, stopped working. Well, in fact, it continued to switch packets, but it become totally unmanageable, nothing better than these lower-end Compex or Surecom switches. Good value for money, isn't it? This brick, truly speaking, never worked fine — its f*cking so-called web-interface is compatible only with IE5, it has no telnet port, it has even no... rs232 interface.

Not willing to pay anything more to D-Link, I decied to fix it myself. At least, I decided, it would be a lot of fun, even if I fail to do that. Furthermore, my boss volunteered to help me out — being a good hardware engineer in the past, he's able to easily fix PCBs using just a 40W soldering iron. So, I rolled up my sleeves, took the screwdriver and opened the box:

The hardware inside was semi-standard for such kind of devices — a printed circuit board with an ARM940T based Samsung S3C2510A SoC mircocontroller as its CPU, forth Vitesse Switch-on-a-chip cores, and a number of Vitesse PHYs (in fact, I thought Vitesse only produce teapots... probably D-Link just missed components — they're hot like a boiled water).

Vitesse folks seems to have a good sense of humor — their PHY chip is called SimpilPHY.

Unfortunately, there was no rs232 connector on the board — thus, no chance to see what happened to it. In fact, there was place to put a standard DB-9 connector on the PCB, but no signal output at all. Clearly, there was something missing... By looking in the CPU manual (well, not the CPU one, but evaluation board manual for previous Samsung model — 2410), I discovered, that the processor uses TTL voltage logic, and output driver for rs232 is recuired. And... near the UART contacts there was a place exactly for MAX2332 UART driver. Looks like they just took their development board and removed all debug components in retail version. Woot, woot!

We took our old oscilloscope to ensure if there were signals at the contact box. Yes, they were!

Next we put the MAX2332 microscheme where it belongs to, and even placed the standard DB-9 connecter to simplify things a bit.

Now the switch board looked like a real ARM development board. I even put a reset button on it:-) However, this didn't helped much. The console attached showed:

uart init successful
flash init successful
Get my mac[0-13-46-36-39-da]

image verify failed, loader mode

Not very informative. In this mode, the console didn't answered any keys, even echo didn't worked. Funny enough, there was one active TCP port on the switches management IP address — port 80, that acted like an echo server. Definitely, something bad happened to firmware.

Fortunately, there was yet one nice connector on the board — JTAG. JTAG stands for Joint Test Action Group, and presents IEEE 1149.1 standard for testing integrated circuits with boundary scans. Nowaday, most SoC manufacturers, implement comprehencive in-circuit debug solutions using JTAG as a serial interface to the chip. In particular, ARM 9 core uses standartized JTAG-based protocol, that allows developer to submit and execute any command on CPU, set breakpoints, and so on. It's not too surprising, that this allows you to do whatever you want with target platform, including reflashing. All you need is to configure a memory controller to map flash properly and implement a program that will run on target hardware and reflash the memory.

In fact, there're a lot of excellent JTAG software available, OpenOCD being among of them. This excellent piece of software can make you feel like debugging a locally runned program while communicating with an embedded processor via JTAG. Using it GDB-server interface you can easily connect your GDB instance to it, and fully control the board. Besides that, it also provide a custom telnet interface, where you can perform a lot of high-level debugging tasks like examining the target memory, modifing it, viewing registers and so on. It also provide interface through which the flash modules can be programmed, and it's easily extensible. It took nothing more than a hour for me to understand the OpenOCD code structure, and perform all required modifications to support D-Link configuration. The switch developers employed AMD AM29LV800 flash chip as it's media — it's a pretty standard chip, so only the identity of the memory and a couple of knobs has to be added to OpenOCD code to support it.

Having nothing else to do, I decided to reflash the switch with recent image available from D-Link. From the processor state it was clear they mapped the flash to 0x80000000 with bootloader occuping the address space up to 0x80030000, and the OS image put just after that address. What's suspicious — D-Link guys configured all eight banks, though only three were really used. Furthermore, all banks were configured in the way as they provide 16MiB of memory each, though it was not true (e.g. flash is only 1 MiB). Have they ever thinked about folks who had to reverse engineer their software in future?;-)

Luckily enough, the switch started working after the reflashing. It happily blinked with all its LEDs, clearly showing how it's happy to be healthy again, displaying on console:

 uart init successful
 flash init successful
 Get my mac[0-13-46-36-39-da]
image verify successful, jump to 0x80030000
cpu init finished 
system init successful 

 flash alloc[222bb8]


web server running... [5307]

Not sure which OS it runs, probably VxWorks or RTEMS. My first assumptions were it'll be some flavour of Linux, but things don't look like this. Probably, some information might be extracted from the image intself, but I didn't looked into this yet. I noticed, however, that the image consists of small unpacker and compressed code. So it has to be unpacked before anything could be analyzed further.

Unfortunately, the chip can't run FreeBSD, or any other BSD as it got no MMU. Thus my dreams about 48-port BGP router have no chances to come true:-(

Anyway, now you know what to do if your switch or router will broke out. Don't pay D-Link extra money for missed functionality — just fix the device yourself. It's easy enough.

Recently I was involved into the practical part of microprocessor's course, included into our department's masters program, where I served as TA. Well, in fact, not just TA, but the entire 2x2 hour course was developed and teached by me and my friend.

The primary goal of this classes was to make students familar with our national processor boards, produced by NIISI (research institute of systems development). Theses boards include our first MIPS R3000-compatible KOMDIV32 microprocessor with technological sizes of 35 microns, "PRIME" boot monitor and various peripheral devices like speaker, uarts, light diods and so on. The processor runs at 90 MHz clock.

The board still very buggy and has a lot of features hardcoded and/or hardwired, but pretty usable. Especially interesting looks PRIME boot monitor, that can handle almost all development task, including but not limited to downloading executables via FTP/TFTP/NFS or Z-Modem, dumping/writing memory and working with FLASH memory. The board has two modes of operation - depending on switch setting it can either go to PRIME on boot, or start executing the code located at the start of flash memory. This feature has shown itself to be very handy in development.

The development kit was also accomplished by the proprietary OS2000 real-time operating system, which seems to be an evil mix of FreeBSD 4.x and VxWorks source code. E.g. their second stage boot loader fully resembles VxWorks loader, though with a lot of parsing bugs. Similary, the kernel object archives contains files like ipfw.ko with full debug info and CVS id's, although it seems to not use it:-) The operating system claims to be POSIX-compatible, though I had no enough time to check it fully. The course was featuring work with some multiprogramming APIs like POSIX threads and message passing, but unfortunately very few studens were able to understand this fully due to lack of C programming experience in past.

We were using FreeBSD as a host enviropment instead of NIISI-recommended Red Hat Linux. All the SDK software was ported to FreeBSD with a little effort (thanks to SDK developers - the didn't used bash/gawk/etc feature much). However, we used their proprietary GCC-based compiler (GPL violation?) under Linux emulation to compile target source code. Probably, gcc's mips target can do that too, but I didn't tried that.

One funny bug we were experiencing - the BSP we got with board was not fully compatible with it, thus it was producing a continuous high-frequency beep after startup. This way we could always say when yet one student managed to get his board running OS2000:-) Although a little room with 8 beeping boards looked slightly crazy...

A good project will be to try to run FreeBSD/mips on these boards and use it as a basis of course, but the lack of documentation and OS source code makes this nearly impossible. Hopefully, they'll understand soon that distributing a research OS without the source code in 21th century is a no-way.

Recently Intel released microcode update for their Core 2 Duo and Core 2 Extreme processors (E6000/E4000 and Q6600/QX6700 series accordingly). This update fix a bug in TLB processing, which can cause incorrect records invalidation. That may result in upredictable system behavour/hangs/reboots/etc.

For FreeBSD you can use devcpu port to update your microcode. It already includes these critical updates, thus will protect you from possible failures linked with the bug. Just install it as usual and add 'devcpu_enable="YES"' in the rc.conf file. Processor microcode will be updated on the next restart (or you can issue /usr/local/etc/rc.d/devcpu start by hand).

Today I've updated the devcpu port. One of the major things that was changed is the utility to deal with Award bios update images was added. This program allows to extract microcode updates from vendor bios updates (this especially important for AMD cpus as this company doesn't distribute cpucodes updates for end-users). Furthermore, you can take bios updates for the newer hardware and extract microcode updates from them in case if your system unsuppored by vendor anymore. A have a lot of such hardware.

Another thing, that was added is the rcNG startup script. It allows to enable automatic cpucode updates on startup. Just add ths string devcpu_enable="YES" in your /etc/rc.conf file and cpus will be updated on each boot.

I wish to think pav@ and netchild@ for their suggestions and reports.