Help with multilevel parallel programming

1 Ansicht (letzte 30 Tage)
Etienne
Etienne am 20 Okt. 2014
Kommentiert: Edric Ellis am 22 Okt. 2014
Greetings All,
I am trying to design a bioinformatics program to perform efficient local (SWA) alignment to a set of 1200 reference sequences over a long list of FASTQ sequences (1-10 million).
I have minimal experience with parallelism but would like to code the individual FASTQ reads to be analyzed in parallel with each analysis taking the individual read and aligning to all the 1200 reference reads in parallel.
Does anyone have any suggestions on how to get around not being able to perform nested parallelism in MATLAB?
Thank in advance.

Melden Sie sich an, um diese Frage zu beantworten.

Antworten (2)

Edric Ellis
Edric Ellis am 21 Okt. 2014
One of the reasons that PARFOR does not support nested parallelism is that it is difficult to achieve good load-balancing in this way. It is usually better to use PARFOR at the outermost level possible (providing there are enough iterations to support a good degree of parallelism). I would advise trying that approach first and seeing where it fails to achieve sufficient speedup.
(Note that you can achieve a kind of nested parallelism with PARFOR either by setting the workers to run in multi-threaded mode, or by using gpuArray in the body of the loop).
  2 Kommentare
Etienne
Etienne am 21 Okt. 2014
Hi Edric,
Thank you for your message. I will need to use the parfor on the outermost loop as it will perform the most iterations (ballpark 1-10 M). The trouble is that the each iteration will need to run the swalign method ~1200 times. This is a bioinformatic alignment method that forms the bulk of the computational complexity.
Can you think of a means of multithreading this alignment step; I can't think of how I can use a gpuArray since this alignment is non-numeric
Thanks,
Etienne
Edric Ellis
Edric Ellis am 22 Okt. 2014
If you've got ~ 1e6 outer loop iterations, PARFOR will be able to spread them over all the workers (i.e. cores) you have, so there would be no benefit to further parallelisation - all your cores will be fully utilised.

Melden Sie sich an, um zu kommentieren.

Etienne
Etienne am 21 Okt. 2014
Hi Edric,
Thank you for your message. I will need to use the parfor on the outermost loop as it will perform the most iterations (ballpark 1-10 M). The trouble is that the each iteration will need to run the swalign method ~1200 times. This is a bioinformatic alignment method that forms the bulk of the computational complexity.
Can you think of a means of multithreading this alignment step; I can't think of how I can use a gpuArray since this alignment is non-numeric
Thanks,
Etienne

Kategorien

Mehr zu Parallel for-Loops (parfor) finden Sie in Help Center und File Exchange

Tags

Produkte

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by