A further update

Was not thinking as clearly as I could have been last time I posted.  The best way to get the contacts is via the ChildLocations property as this will tell you the relationship type, for example:

Records request = new Records();
request.q = "REC_349";
request.Properties = new PropertyList("ChildLocations");

RecordsResponse response = trimClient.Get<RecordsResponse>(request);

foreach (var rec in response.Results)
{
    foreach (var childLoc in rec.ChildLocations)
    {
        Console.WriteLine("{0} - {1}", childLoc.Name, childLoc.TypeOfContact);
    }
}

Update

As noted by Michelle below you cannot add a contact that already exists.  The way to find who is already a contact is to do a Location search, for example:

Locations request = new Locations();
request.q = "contactOf:REC_1";
LocationsResponse response = trimClient.Get<LocationsResponse>(request);
foreach (Location loc in response.Results)
{
    Console.WriteLine(loc.Uri);
}

The problem

Contacts have a slightly more complex relationship to Records than is ideal, for example they have a one to many relationship but are not represented by a child list.  You can set the 'Other' contact using Record.OtherContact but then how do you more contacts?

When using the ServiceAPI look in these four places when trying to get or set data:

1. the stock properties (e.g. Record.Title, Record.OtherContact),
2. child lists (e.g. Record.ChildHolds)
3. custom properties (e.g. Record.Html - check the documentation for which of these are read only), and
4. service actions.

Service Action to set multiple contacts

Service actions are only for writing, not reading.  Reading is done via properties.  You may call the same service action multiple times in a single request and the ServiceAPI help contains a page to demonstrate the JSON for every action.  Below is the C# code to add multiple contacts to a Record.

Record rec = new Record();
rec.Uri = 9000000001;
rec.ActionsToCall = new List<IMainObjectActionRequest>();

rec.ActionsToCall.Add( new AttachContact()
{
    AsContactType = ContactType.Addressee,
    ContactLocation = new LocationRef() { FindBy = "Me" }
});

rec.ActionsToCall.Add(new AttachContact()
{
    AsContactType = ContactType.Other,
    ContactLocation = new LocationRef() { Uri = 9000000072 }
});

trimClient.Post<RecordsResponse>(rec);