I think a lot of people have, at some point, had a memory card that has failed on them. Most of the time, due to their ubiquitous nature, it is going to have been some variation of an SD card, most likely a microSD card.
Unlike most people, I have always kept my broken memory cards in a little bag in the hopes that one day, in the far future, I might be able to get them working again. And slowly, over time, I have reclaimed nearly all of my "bad cards" from that little bag of despair and put them back into usage. All but one.
I have had a component in the box for reading SD cards for a while now, and the desire to do something with an ESP32 took me, and I wondered if I could get it hooked up to analyze my card.
At this point, nothing I have tried has ever been able to do anything useful with this card since it failed. Every device and operating system it has been in has either said it cannot be read, or doesnt even detect it. So I thought it was headed for the bin, it even had the dreaded permanent black marker spot of doom - the precursor for all my stuff that just has to be scrapped.
But, ever the optimist, I kept hanging on to it, after all it is so small...
The component in question is called a Micro SD Sniffer. It is essentially a bare MicroSD card reader with a passthrough that comes with a breakout for the pins. This allows you to pop a MicroSD in to the slot, then plug that into a host device. The breakout allows you to attach an oscilloscope to the lines from the card itself, so you can essentially see the traffic as the connection is established etc. It can help you establish if there are any problems, but it is much better as a learning tool in my experience.
This isnt an SPI SD Card breakout though, these are bare connections to the SD card itself, so this is a consideration you have to keep in mind when wiring this up.
 |
| the Micro SD Sniffer Passthrough |
As you can see from the image above, this has a lot more connections when compared to a regular SD card breakout board. This allows you to configure the component for many different for both SD and SPI mode directly. It turns out that all SD cards have two modes of operation, SD mode and SPI mode. SD mode allows for faster transfer rates and for things like SDIO, whilst SPI is a lot slower - but is almost universally supported by everything that has a memory card slot.
My doomed card is a Sandisk Ultra 64GB MicroSDXC 1 card, apparently, so this should win SD mode, but to begin with I am going to take a look at it in SPI mode - just to make sure I can read something from it.
Because nothing can read this in a normal fashion, I want to see what will happen when it is just told to do something, like return its manufacturing information, I can get that on a desktop operating system for example. To do this, I need to understand how SD cards work, and how they communicate. Thankfully, there is a very helpful set of lecture notes on this exact topic here: SD Card Lecture Notes that describe the process in which you can send raw SPI commands to an SD card and communicate with it over the SPI bus. It gives you complete examples on how to get the card to start listening, send commands and understand the response.
In addition to this, there is another great set of examples, complete with pseudo-code that helped me out a lot available here: https://nodeloop.org/guides. It provides all the information you need to create something that will send raw SPI commands over to an SD card.
One thing to point out here is that the Micro SD Sniffer isnt compatible with all micro-controllers, for instance this wont work correctly with an ESP8266 without modification.
Using a generic ESP32 development board, I can connect the SD sniffer to the SPI bus using pin 5 for chip select like this:
Using Arduino as the platform, we can actually send raw SPI commands to the ESP, together with the information from the references I mentioned above, I can create the raw SPI commands I need to speak to the SD card and see if it responds.
To keep things clear, and focus on the architecture, the code snippets below are simplified to show the core logic. If you are following along yourself, you will need to adapt them into your own Arduino sketches. During the process of creating this, I went ahead and created a class stored in a separate header file for almost everything you will see below. The actual files I created will be available up the blogs GitHub here: https://github.com/1rafayal
If we take a look at the lecture notes from earlier again, we can see that the structure for an SD card command always starts with a 01, and is followed by a six bit command number. So, by creating an 8 bit value with 0x40 we start of with something we can work with. I am achieving this with the following line:
In order to even start speaking to the card, there are a number of things we are going to need to set up in the code first. As I am using a NodeMCU ESP32s clone here, I am going to go ahead and define the SPI pins. This will make it easy for anyone who wants to try this out for themselves. I am also going to go ahead and enumerate the SPI commands I need to use later on to make things a little easier to follow through:
#define PIN_CS 5
#define PIN_SCK 18
#define PIN_MISO 19
#define PIN_MOSI 23
enum SDCommand : uint8_t {
CMD0 = 0, // GO_IDLE_STATE
CMD8 = 8, // SEND_IF_COND
CMD9 = 9, // SEND_CSD
CMD10 = 10, // SEND_CID
CMD41 = 41, // SD_SEND_OP_COND (ACMD41)
CMD55 = 55, // APP_CMD
CMD58 = 58 // READ_OCR
};
The next thing I need to do is initialise the card itself. To do this, I am going to need some way of sending commands to the card in a structure that it will understand. Because I am going to be sending over some raw SPI commands, I need to define some SPI settings to use. The best way to manage this is to implement an SPI transaction.
So with this in mind, I first need to set up my SPI transaction and prepare to initialise the card itself. I can do that with the following:
bool begin() {
SPI.begin(PIN_SCK, PIN_MISO, PIN_MOSI, _csPin);
pinMode(_csPin, OUTPUT);
digitalWrite(_csPin, HIGH);
SPI.beginTransaction(_spiSettings);
bool ok = initCard();
SPI.endTransaction();
return ok;
}
Next, I start the SPI transaction itself, this requires some actual settings that are specific for communicating with an SD card. In order to speak to the SD card, we need to specify the clock speed, bit order and SPI mode. To do this, we can create an instance of the SPI settings like this:
SPISettings _spiSettings;
According to the notes I linked above, SD cards have clock speed of 400Khz, require the most significant bit first and use SPI mode 0. To define those settings in the code, I can do the following:
_spiSettings(400000, MSBFIRST, SPI_MODE0)
The first thing we do in initCard() is send those 80 clock pulses to get the card ready and listening for commands. In the code, it looks like this:
for (int i = 0; i < 10; i++) SPI.transfer(0xFF);
From here, we need to reset the card into SPI mode. we can do this by sending over the first SPI command in our list - CMD0:
uint8_t r = 0xFF;
for (int i = 0; i < 5 && r != 0x01; i++) {
r = sendCmd(CMD0, 0, 0x95);
}
if (r != 0x01) return false;
But, sendCMD isnt a standard library, its a function that I made to help create the SPI commands that I want to send over to the SD card, so I should probably talk about that before going further forwards.
The purpose of this method is to collect various values and construct an SPI command that an SD card is capable of understanding. According to the notes above, "communications with an SD card is performed by sending commands to it and receiving responses from it". The structure of the command is 48 bits long, from looking ant initCard() above, you can see that I am not supplying any bytes when calling the method - the method takes those arguments and constructs our command for us. So far, we have already sent the command for the card to be reset into SPI mode, so sending CMD0 should return something when a working card is used.
The purpose of this method is to collect various values and construct an SPI command that an SD card is capable of understanding. According to the notes above, "communications with an SD card is performed by sending commands to it and receiving responses from it". The structure of the command is 48 bits long, from looking ant initCard() above, you can see that I am not supplying any bytes when calling the method - the method takes those arguments and constructs our command for us. So far, we have already sent the command for the card to be reset into SPI mode, so sending CMD0 should return something when a working card is used.
Lets take a look at how I constructed sendCMD to construct the SPI commands:
uint8_t sendCmd(uint8_t cmd, uint32_t arg, uint8_t crc, uint8_t *tail = nullptr, uint8_t tailLen = 0) {
digitalWrite(_csPin, LOW);
SPI.transfer(0xFF); //this syns up the clocks.
SPI.transfer(0x40 | cmd);
SPI.transfer(arg >> 24);
SPI.transfer(arg >> 16);
SPI.transfer(arg >> 8);
SPI.transfer(arg);
SPI.transfer(crc);
uint8_t r = 0xFF;
for (int i = 0; i < 10; i++) {
r = SPI.transfer(0xFF);
if (!(r & 0x80)) break;
}
for (uint8_t i = 0; i < tailLen; i++) {
tail[i] = SPI.transfer(0xFF);
}
digitalWrite(_csPin, HIGH);
SPI.transfer(0xFF); //extra clock byte so the card releases the MISO line
return r;
}
Next, I bring the CS pin low - this enables the SD card. From here I start to send over individual bytes via SPI. The first is required to sync up the clocks between the card and the ESP 32, the remaining transfers are sending over the bits that make up the SPI command itself.
It does this by taking the parameters we used when calling the method and performing some bitwise maths. The reason why I have to do it this way is both complex and simple. SPI is what it name states, it is a serial interface. When we use transfer() using the SPI library, each bit is sent from the ESP to the SD card, which places those bits into a specific register. The card itself is able to determine where to store these bits, it has to act on the command and the argument it has received and provide a response - if it can. We have already told the card to work in SPI mode, so it knows to treat incoming bits as SPI communications.
It does this by taking the parameters we used when calling the method and performing some bitwise maths. The reason why I have to do it this way is both complex and simple. SPI is what it name states, it is a serial interface. When we use transfer() using the SPI library, each bit is sent from the ESP to the SD card, which places those bits into a specific register. The card itself is able to determine where to store these bits, it has to act on the command and the argument it has received and provide a response - if it can. We have already told the card to work in SPI mode, so it knows to treat incoming bits as SPI communications.
If we take a look at the lecture notes from earlier again, we can see that the structure for an SD card command always starts with a 01, and is followed by a six bit command number. So, by creating an 8 bit value with 0x40 we start of with something we can work with. I am achieving this with the following line:
SPI.transfer(0x40 | cmd);
From here, we need to send the rest of the command, so far we have only sent the first 8 bits - there is still another 40 bits to go.
The next part of the command deals with any arguments that are supplied in the parameters. These are supplied as unsigned 32 bit integers, however transfer() requires a an unsigned 8 bit integer. To get the arguments I perform a number of bitwise shifts to select the byte I actually want to send. For the first of these operations, I perform a bitwise shift moving everything to the right by 24 places. This discards the lowest 24 bits - they get binned entirely and only the remainder is kept, leaving us with one byte, the most significant bit, which is moved down into the lowest 8 bits.
And this is what transfer() is going to try and send.
The final transfer doesnt require any shifting as I only need to send the last byte of the integer.
The last part of the command transferred over is the CRC, and requires no manipulation. Using the references I mentioned above, I discovered that the CRC's for CMD0 and CMD8 are required and are unique. However, from that point on, it seems like you can use a generic CRC for each command.
After we have fired the command over to the card, we can expect a response back from it. This is known as the R1 response. There are eight R1 responses that can be returned by the card itself:
| Bit | Meaning when set |
|---|---|
| 7 | always 0 |
| 6 | parameter error |
| 5 | address error |
| 4 | erase sequence error |
| 3 | CRC error |
| 2 | illegal command |
| 1 | erase reset |
| 0 | card is in the idle state |
Once sendCMD has finished transferring its bytes, I listen for the response on r, setting it to a default value of 255. I have done this because it matches none of the R1 responses above, so if r is still 255 after the command has been sent, then there is an issue and we can break out from the function here.
If not, then we can go ahead and read the rest of the response bytes. The method finishes up by setting the CS pin high, disabling the SD card and releasing the MISO line and as a result of this we are given the entire response from the card.
However now, I am at the point where I can set up, initialise and send commands to an SD card - and best of all, I should expect a response with a known working card. This assumes that I have the bitwise maths correct.
Spoiler, I do.
Spoiler, I do.
Now that I have initialised the card, and even retrieved some information from it, I want to go ahead and get the properties of the card itself, for want of a better term. I am specifically interested to find out the reported capacity and if there are any locks or errors currently present. I know that SD cards can go into a read only mode internally, but I am also interested to see how I can get the card to report its capacity, as opposed to detecting the free space.
Running sendCMD has already set the card up and populated it with the answers I need, I just need to go ahead and retrieve it. Previously, running that method returned an R1 response bit in hex. However, it can return a lot more information spread across multiple bits. So just like we read them out, we need to read them in.
To print out the card info, we need to start a new SPI transaction - dont forget that the previous actions have left the SD card in the state we want it in, but we ended the SPI transaction at the end of that operation. We dont need to worry about the settings, we are going to carry on using the same ones, so initiating the transaction has not changed. I created a method called printCardInfo() to control this operation:
void printCardInfo() {
SPI.beginTransaction(_spiSettings);
uint8_t csd[16], cid[16];
// read the CSD, then work out the capacity for whichever format it is
if (!readRegister(CMD9, csd)) {
Serial.println("CSD read failed (see R1 data token.");
} else {
printHex("CSD Register: ", csd);
uint8_t csdVer = csd[0] >> 6;
Serial.printf("CSD structure version: %u\n", csdVer);
uint64_t capacityMB = 0;
if (csdVer == 1) {
// CSD v2 (SDHC/SDXC): capacity = (C_SIZE + 1) * 512 KB
uint32_t cSize = ((uint32_t)(csd[7] & 0x3F) << 16) | ((uint32_t)csd[8] << 8) | csd[9];
capacityMB = ((uint64_t)cSize + 1) / 2;
} else if (csdVer == 0) {
// CSD v1 (standard capacity, <= 2 GB)
uint32_t cSize = ((uint32_t)(csd[6] & 0x03) << 10) | ((uint32_t)csd[7] << 2) | (csd[8] >> 6);
uint8_t cSizeMult = ((csd[9] & 0x03) << 1) | (csd[10] >> 7);
uint8_t readBlLen = csd[5] & 0x0F;
uint64_t bytes = ((uint64_t)(cSize + 1)) << (cSizeMult + 2) << readBlLen;
capacityMB = bytes >> 20;
} else {
Serial.println("Unknown CSD structure version.");
}
if (capacityMB) {
Serial.printf("Capacity: %lu MB (~%d GB)\n", (unsigned long)capacityMB, (int)(capacityMB / 1000));
}
}
// reads the CID
if (readRegister(CMD10, cid)) {
printHex("CID Register: ", cid);
Serial.printf("Manufacturer ID: 0x%02X\n", cid[0]);
Serial.printf("OEM ID: %c%c\n", cid[1], cid[2]);
Serial.print("Product Name: ");
for (int i = 3; i <= 7; i++) Serial.write(cid[i]);
Serial.println();
uint32_t sn = ((uint32_t)cid[9] << 24) | ((uint32_t)cid[10] << 16) | ((uint32_t)cid[11] << 8) | cid[12];
Serial.printf("Serial Number: 0x%08lX\n", (unsigned long)sn);
} else {
Serial.println("Failed to read card identity from CID.");
}
SPI.endTransaction();
}
Now, lets take a look at readRegister():
bool readRegister(uint8_t cmd, uint8_t *buf) {
digitalWrite(_csPin, LOW);
SPI.transfer(0xFF);
SPI.transfer(0x40 | cmd);
for (int i = 0; i < 4; i++) SPI.transfer(0x00);
SPI.transfer(0xFF);
uint8_t r = 0xFF;
for (int i = 0; i < 10; i++) {
r = SPI.transfer(0xFF);
if (!(r & 0x80)) break;
}
bool success = false;
Serial.printf("CMD%u R1 -> 0x%02X (Expected: 0x00)\n", cmd, r);
if (r == 0x00) {
uint32_t start = millis();
uint8_t token;
do {
token = SPI.transfer(0xFF);
} while (token == 0xFF && millis() - start < 200);
Serial.printf("CMD%u data token -> 0x%02X (Expected: 0xFE)\n", cmd, token);
if (token == 0xFE) { //this is our data ttoken
for (int i = 0; i < 16; i++) buf[i] = SPI.transfer(0xFF);
SPI.transfer(0xFF); //drop two bytes from the CRC across these lines.
SPI.transfer(0xFF);
success = true;
}
}
digitalWrite(_csPin, HIGH);
SPI.transfer(0xFF);
return success;
}
What makes this different is that we are passing in our array of unsigned integers each time we are calling readRegister(), and each time that logic is executed, it will try to populate that array with a response from the card. This all takes place within this block:
if (r == 0x00) {
uint32_t start = millis();
uint8_t token;
do {
token = SPI.transfer(0xFF);
} while (token == 0xFF && millis() - start < 200);
Serial.printf("CMD%u data token -> 0x%02X (Expected: 0xFE)\n", cmd, token);
if (token == 0xFE) { //this is our data ttoken
for (int i = 0; i < 16; i++) buf[i] = SPI.transfer(0xFF);
SPI.transfer(0xFF); //drop two bytes from the CRC across these lines.
SPI.transfer(0xFF);
success = true;
From there, we loop over the response, one bit at a time and add it to buf each time the loop is incremented. Our array holds 16 elements and we are only expecting 16 bits as a response, plus a CRC. We can handle the CRC by transferring anything and ignoring the response.
PrintCardInfo() does this twice, once for the CID and once for the CSD. There is a reference to an additional method called printHex() which I used to help format and print hex out as part of the responses:
I didnt just write all of the code out in one go, I did iteratively test it as I was going to make sure I was doing it correctly, especially with the bitwise maths. The end result with a known working card looks like this:
This shows a working 2GB card along with all of the data that the card reported itself. I think it is important to note that the ESP didnt calculate any of this - the card itself reported it through its own embedded systems. But this is a known working card - how does my poor little 64GB card look when it is put through the same process?
Well, this card refused to be read by pretty much everything I have ever tried it in since it first died. This is the only thing that has every gotten anything from it since then. As an aside, Windows now actually sees the same thing as my ESP32 reports, but it is still broken.
The card itself reports that it has a 30MB capacity, somewhat less than 64GB. The serial number also appears to be all zeroes - also not a good sign. The manufacturing data also seems to be off.
A quick check on Google sheds some more light on this, apparently SanDisk cards exhibiting this type of behavior is not a rare thing. It demonstrates that the controller chip on the card itself can no longer communicate with the NAND memory. Going into a default state where only a small partition is displayed to the user that cannot be used.
After keeping hold of this little card for all of those years, I am unable to do anything to bring it back to life. However, I have learned a great deal on the topic of SD cards, and how they can work. And what I have covered here isnt the extent of what you can do with an SD card, there are more commands available to work with. As well as SD mode which is entirely separate from SPI mode.
One thing I regret about this activity is not having a scope that would have let me see the communication take place between the card and the ESP itself, I have two basic ones but they didn't capture the visualizations I wanted.
The most interesting, and useful part of this was working directly with the SPI bus. I took a lot of inspiration from looking at existing libraries for SPI devices as well as the Arduino documentation for SPI, this coupled with the indispensable lecture notes I found are what made this possible, so a massive thanks goes out to those guys for publishing that information. Also, this isnt something that is specific to the ESP32, with some adaptation this will also work with any micro-controller that is supported by the Arduino platform., the Pico would be a great device to use if you dont have an ESP32 to hand, all you need to do is change the pins.
I am definitely going to come back to this device to try and test out some different things. And I am definitely going to try it out with other micro-controllers.
I am definitely going to come back to this device to try and test out some different things. And I am definitely going to try it out with other micro-controllers.
