Archive for the ‘Software’ category

« Starting home page

Virtual Reaction Game — developing brave new worlds

July 2, 2015

You are suddenly riding around Stanford in an Aston Martin convertible.  3D text appears in front of your car with game instructions and a countdown timer.  Ringing bombs spawn all around you; you must locate and defuse them!  If you run out time, you should restart the level… or continue playing amongst colorful exploding bombs which will certainly induce nausea.

This truly describes the premise and experience of Virtual Reaction, an immersive Virtual Reality (VR) game.  My team created this interactive training game to improve a user’s sensory processing and reaction times.  We hypothesize that Virtual Reaction maximizes users’ stimuli and engagement which results in better training and more fun.

arcade

Explaining Virtual Reaction to my friend at the Synapse Arcade. He was the first of only two people to beat the game’s hard level!

New Skills:

  • Unity 5 Game Engine.  Javascript/C# scripts
  • Developed VR environment with Oculus Rift DK2 featuring live-action footage

Next Questions:  During the game’s development, I encountered 4 salient problems that I think epitomizes VR’s biggest challenges:

  1. Lagging Frame Rate:  Our game needed to be more efficient with its computation.  There was noticeable lag when changing your view.
  2. Un-immersive Sound: multi-sensory integration is essential to make our surroundings believable.  Our game featured background music, driving/car/wind sounds, and ringing bombs in 3d space.  When mixed together, the sound wasn’t convincing.
  3. Unreal interactive experience:  without Oculus Touch, input was constrained to computer keys.  Typing does not translate into VR well so users may feel disoriented trying to interact with the game.
  4. VR sickness: the 3 previous issues and several other  factors caused discomfort for users.  Being in VR is always fun, however during its development I required frequent breaks in order to rest my eyes and keep my dinner inside me.

Key Takeaway:  VR will disrupt, enhance, and redefine how we consume media.  Applications are everywhere: gaming, cinema, training, research experiments, surgery, defense, communication, and a lot more.  Right now, it’s a young evolving platform that’s experiencing significant growing pains.  But I believe it’s only a matter of time before people overcome these challenges and unlock VR’s potential.

Screen Shot 2015-07-02 at 2.53.15 AM

Our poster explains the basis of the VR game as a neuroplastic training environment.

Categories: Gaming, Software, Virtual Reality

Comments: Be the first to comment

Software — it’s icing on the cake!

July 2, 2015

After completing most EE requirements and electives, I became comfortable with the department’s quirks, smaller classes, familiar classmates.  So it was a shock in my last year to go through the Computer Science introductory sequence.  It was like beginning college all over again – huge classes with hundreds of Freshmen, a new set of rite-of-passage hurdles.  The best assignments were:

  • submitting a blackjack game for a coding competition
  • creating Boggle
  • defusing a ‘bomb’ code within a morass of machine instructions
  • creating a heap allocator, which is the way a computer organizes memory for the user
  • implementing part of a file system
  • managing an army of computer threads to gather an Internet news search. [Might do individual posts on all of these later…]

New Skills:  C systems programming.  C++, Java object-oriented programming.  Python, Linux familiarity.

Key Takeaway:  Building computer hardware is like baking cake.  It’s fun, but it’s more scientific than personal.  It requires expertise because an error can make the cake a disaster.  Figuring out the right mixture of flour, sugar, butter, and eggs.  Balancing flavor extracts or other ingredients for the specific type of batter.  After it is baked (..or fabricated), it belongs to the software guys who decorate the cake and bring it to the party!  First they organize the layers.  Then put fruit, frosting, candles, and write a message based on the occasion (..or application).  Their work is more personal, more creative, closer to the end-user.  It’s quick and messy, but if they screw up, they just wipe it off and start over.

At the party, consumers only see the cake’s presentation, but come to appreciate it fully when eating it.  I don’t yet know where I’ll be in this process, but I know I’ll always love cake.

Categories: Software

Comments: Be the first to comment

VisualTouch Music Game — blending HW & SW

July 1, 2015

Embedded systems are everywhere.  A digital watch, microwave oven, electronic toy, and your car dashboard are a few disparate examples.  The characteristics of an embedded system are:

  • Made from a combination of hardware and software
  • Task specific. They’re designed to perform one specialized task as opposed to general purpose computers
  • Usually reactive or feedback-oriented.  E.g. most home or kitchen appliances only function when you interact with them.
  • Built with efficiency and frugality.  Your toaster needs a timer and heat control, but it really doesn’t need a full-fledged CPU
  • Should be reliable and stable.  Because they are simple, we expect these machines to work. They shouldn’t require maintenance because it’s often easier and cheaper to buy a new one.

For my required EE Hardware design project, my team developed an embedded system on an Atlys Spartan-6 Development Board.  This board constructs your hardware specification, like an FPGA, but then it can also run C software that you place on top of the hardware.

StanleyFPGA

My brother’s Flat Stanley visited me while I was working in the EE lab.  Sadly, I couldn’t find a photo of our finished project, but this is our Spartan-6 Development Board

We hooked up a webcam, a screen, and speakers to our Board.  We had the screen display the camera’s output.  Our game consisted of these playable modes:

  1. Changes the chord’s duration based on the red/blue/green content of the overall pixel count.
  2. Changes the chord based the amount of movement detected by the camera.
  3. A grid is overlaid on the screen. When a user in front of the camera covers a square of the grid with their hand, the board plays a song.

Skills:

  •  Verilog code in Xilinx Platform Studio configures the processor-based system.  [The hardware platform managed the transfer of data from the camera to the screen.  All the song data was also implemented in hardware.]
  •  C programming in Xilinx Software Development Kit.  [Everything else:  the music player, game controls, game logic, pixel computations were done in software.]

Key Takeaway:    This project was a turning point: creating the hardware infrastructure was important, but the software was the crux of the game! My specialty was more on the hardware side and I realized that I shouldn’t leave college without delving into the software domain.

Screen Shot 2015-07-01 at 8.00.14 PM

DJ mode allows user to mash these pop songs together based on where one’s hand is positioned in the screen’s grid.

Categories: Gaming, Hardware, Music, Software

Comments: Be the first to comment

CPU Optimization — a neverending story

July 1, 2015

EE282 Computer Systems Architecture introduced me to more complex CPUs (microprocessors are named Central Processing Units when packaged within complete computers) and to more advanced techniques for architects building CPUs.  There were two big optimization projects:

  1. Optimize Matrix Multiplication:  We had to implement optimization techniques in C to achieve faster matrix multiplication.  A very effective technique is loop unrolling which basically puts more work in each loop iteration to reduce the number of iterations. For example, a for-loop executing a single multiplication needs 100 iterations; the optimized version only need 10 iterations, but each iteration grabs a bigger chunk of data and does 10 multiplications in each loop around.
  2. Optimize Mystery Computer:  There’s a dynamic mystery computer that executes large mystery computations.  We could remotely control the computer’s number of processors, processor speed, and cache system.  We then had to explore paths to an optimal computer configuration among thousands of possibilities.  My partner and I were like Sherlock and Watson; discovering new clues, having eureka moments, and then surmising explanations afterwards.

New Skills:  C programming, performance analysis, and optimization techniques

Key Takeaway:  The official Olympic motto is Citius, Altius, Fortius which is Latin for ‘faster, higher, stronger’.  I think this motto’s underlying sentiment isn’t to celebrate the fastest, highest, strongest but to spur athletes toward continuous improvement.  If CPU development had its own motto it would be faster, smaller, less energy.

Ever since their invention, new CPUs are released that are more powerful, with smaller transistors, and more energy efficient.  And just like there’s no perfect athlete or sports team, there’s no ultimate computer because progress in one area comes at the expense of others.

Categories: Hardware, Optimization, Software

Comments: Be the first to comment

Arduino Music Box

June 30, 2015

In my first year, I took an introductory electronics course that gave us access to

  • analog circuits:  circuits are closed loops featuring electronic components (such as resistors, transistors, LEDs, transistors, amplifiers);  there’s also a power source that raises the voltage, pumps current, putting all the components in action.
  • sensors:  interactive devices (such as buttons, wheels, temperature sensor etc. ) that can sense their surroundings
  • microcrontrollers:  named ‘micro’ because they are small, simple, cheap computers; ‘controller’ because they are mainly are used to run simple logic to control other devices.  Though it can execute code like any computer, it doesn’t have the computing power to run your laptop or phone.

After training us with all these parts and more, we were then let loose to build our own final project.  I had the most fun running simple code on an Arduino Nano microcrontroller.    Using an empty cigar box, I made a music box that plays mp3s from an SD card and displays its song name on a tiny screen.

This was filmed in 2011 when my phone was technologically more similar to a potato than a modern smartphone.

New skills:

  • First time constructing non-trivial circuits and using equations like Ohm’s Law in real life instead in a notebook.
  • C++ programming in using Arduino Integrated Development Environment (IDE) which runs the code.

Followup Questions:  How does the microcontroller work?  How did it interpret the mp3 file and decode into playable sounds? How to organize circuit wirings so it’s not a hot mess?

Key Takeaway:  An auspicious introduction to electronics where I quickly learned that digital domain (running code on microcrontroller) is friendlier than analog domain (soldering little wires and devices together and measuring with oscilloscope).

In retrospect, this key takeaway became a catalyst for a lot of other things.  First impressions DO count!

Categories: Hardware, Music, Software

Comments: Be the first to comment