A question for feeding moms only?

My strictly fed 5-month old was diagnosed with a cow-milk allergy after an extreme allergic reaction 2 weeks ago. As much of a pain as it is, I cut all dairy out of my diet to continue feeding him. On Monday he was admitted to the hospital on for bronchitis, vomiting, dehydration, rota virus, and a list of other things. He hadn't fed for about 2 days (horribly painful, btw!) and in that time due to various cirstances i was only able to pump 1 time. Because he was still not eating much in that time my body quit making much milk at all. I have spent the past 3 days doing everything my doctor recommended to increase milk supply, but it doesn't seem to be working. It's very frustrating to both me and the baby and making both of us fairly miserable. my doctor said not to stop feeding but in the same breath said I could try some soy formula if i wanted. I know plenty of babies who thrive just fine on formula, but I just didn't see my little guy being one of them. Do you think I should hold out and keep trying to increase supply or should I just accept "defeat" and give him formula?