I have thought quite a bit about these ideas myself.
you could do embedded pc or netbook with linux + sampler software. (http://www.linuxsampler.org/about.html)
Tuned up correctly or with the right hardware latency should be a non issue.
Sure you could fully DIY a drum sampler.. and if you had a million of them manufactured it would be way cheaper, but the reality is you get more for you money to use mass produced hardware.
Now if you were only interested in synthesised drum sounds it makes things a lot simpler (no need for masses of RAM etc). There are already DIY solutions for this... MIDIbox (http://www.midibox.org/dokuwiki/doku.php?id=midibox_sid)
but its a lot of time and money for a limited capability in the context of drum sounds
or as everyone else has said.. just get a drum machine..or an old drum module and drive it with megadrum.
Some informed discussion on it here:
http://www.vdrums.com/forum/showthread.php?t=58143