[Halld-offline] genr8 software request

[Richard Jones]

Mark,

This issue of how random number generators are initialized deserves a more general discussion, leading to an agreed policy, I think.  I also ran into this recently when I submitted a couple thousand parallel simulation jobs on the OSG, and about 30% of them ended up duplicating a seed used by another job.  Again, I traced it down to the outdated practice of using unix time() to seed the random number generator.  I say "outdated", although I think there never really was a time when it made sense to do that, except for run-once-and-throw-away software projects.

All of our applications that use random numbers should have a command-line option for passing in the seed, but that only begs the question.  What is a good source of seeds for Gluex software in need of pseudo-random sequences?  I can think of several good options we might consider:

   1. /dev/random - this is a pretty good attempt by the linux kernel folks to provide us with true random numbers, for things like secure key generation where it really matters.  It collects entropy from the OS environment and keeps track of how much entropy it spends each time it makes a random number for you.  The down side is that you can easily drain the entropy pool, after which your reads might wait for quite a while for the entropy to build up again, depending on how busy the system is.  You open and read from /dev/random just like any file, but don't read more bytes than you are going to use, because it depletes the pool.
   2. /dev/urandom - this behaves like /dev/random, except that it never blocks.  If it runs low in entropy then it just continues to supply pseudo-random numbers, in the spirit of the pseudo-random number generators used internally by our software.
   3. a web service, similar to the one provided by http://www.random.org.  Of course we do not require all of the statistical properties guaranteed by random.org, and we don't want to pay their fees.  But we could make our own web service hosted on a jlab server and replicated at other sites to ensure high availability, and make a command-line tool that grabs a given number of bytes and spit them out on standard out.  Using this service might look something like "$ genr8 -s `get_gluex_randoms -n 2 -f '%d' ` ..."

If one wanted to be able to run "off the grid", the command "get_gluex_randoms" could fail over to /dev/random or /dev/urandom if the internet could not be reached.

Comments?

-Richard J.






On 12/3/2010 8:25 AM, David Lawrence wrote:
> Hi Kei,
>
>       I'll modify the program to allow the seed to be set explicitly via
> command line argument. I'll let you know when the modification is available.
>
> Regards,
> -David
>
> On 12/2/10 4:45 PM, Kei Moriya wrote:
>> Dear Offliners,
>>
>> I have been working on the Hall D simulation packages,
>> and have encountered an issue with genr8, and was wondering
>> if anybody could help me.
>>
>> The issue I encountered is that the seed used for the
>> random generator in genr8 is the current time,
>> so if the program is run consecutively within the same
>> second, the generated result is exactly the same, and I end
>> up with several copies of the same files.
>>
>> I think that the use of the current time should be avoided,
>> not just because of the behavior above, but also because
>> it becomes impossible to reproduce the same file at a later
>> time if necessary.
>>
>> I think passing the seed in as another runtime parameter
>> to genr8 would require only a few lines of extra code.
>> Could somebody do this for me, or point me to how to update
>> the program myself?
>>
>> Thanks in advance.
>>
>> 	Kei Moriya
>> _______________________________________________
>> Halld-offline mailing list
>> Halld-offline at jlab.org
>> https://mailman.jlab.org/mailman/listinfo/halld-offline
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailman.jlab.org/pipermail/halld-offline/attachments/20101204/bc5971e5/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4092 bytes
Desc: S/MIME Cryptographic Signature
Url : https://mailman.jlab.org/pipermail/halld-offline/attachments/20101204/bc5971e5/attachment.bin