[Rsiena-help] RSiena on a computer cluster

[Tom Snijders]

Dear Tobias,

The multicluster option in RSiena is based on the R package parallel. It has the disadvantage of requiring rather much communication between the processors. How this works out in practice depends strongly on the hardware configuration. In my experience, using multiple processes does have an advantage over the use of only one process. I would guess that a really large number makes no difference, and 16 seems already a quite large number in this respect. The result that using 8 processes takes more time than 1, and 16 takes less time, seems to me totally hardware specific.
But I do not know a lot about this, and if anybody else can correct me or say more specific things, that would be great.

We are still hoping that the settings model will be implemented some time in the future, which should be much more reasonable and less time-consuming for large networks. But this is not yet nearby.

Best wishes,
Tom


================================================================
Tom A.B. Snijders
Professor of Statistics in the Social Sciences
Department of Politics and Department of Statistics
Nuffield College
University of Oxford
tel. +44-01865-278599


-----Original Message-----
From: rsiena-help-bounces at lists.r-forge.r-project.org [mailto:rsiena-help-bounces at lists.r-forge.r-project.org] On Behalf Of Tobias Stark
Sent: 18 September 2013 06:55
To: rsiena-help at lists.r-forge.r-project.org
Subject: [Rsiena-help] RSiena on a computer cluster

Dear RSiena developers,

I hope to increase the speed of my analyses using a computer cluster. I ran the exact same test analysis with a large network (approx. 1,000 nodes) and varied the number of cores on which SIENA could run. I noticed that there was hardly any gain in speed using more cores. In fact, the analysis took longer when I ran it on 8 cores instead of 4 cores (no matter if the cores where on one machine or distributed across the cluster). The analyses where considerably faster on 16 cores but using 26 or even 32 cores did not result in quicker results

I wonder if there is a restriction within SIENA that prevents additional gains in speed with more cores or if the problem lies with the communication between machines in the computer cluster. Do you have a hint for me?

Thanks,
Tobias
_______________________________________________
Rsiena-help mailing list
Rsiena-help at lists.r-forge.r-project.org
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rsiena-help

Nuffield College is a Registered Charity No. 1137506. Registered Office: Nuffield College, New Road, Oxford, OX1 1NF