C :: Compact Table Representation - Data Structure

Data structure problem. I have a table of the following format:

Code:
C/R 1 2 3 4 5 6 7 ...
1 x x x
2 x
3 x
4 x
5 x x x
6
... s

So my column and row names are integer numbers if an outcome of some game for a certain column and row lable is a match then we have an x on that position. The size of the integer names for both column and row name is quite large (let us imagine that it is so large that you would need a machine with 500GB of RAM memory to hold this type of table even if x's are treated as regular char's) . In every row there is at least one x and for every column the same holds for the columns. However, the number of x's for a row or a column can be bigger then 1. How to store this table efficiently? (using as less memory as possible).

The data structure should be efficiently accessed in the column fashion that is, if i want to get all values for column 4 I should be able to do that in O(N) time where N= the number of rows.