How many concurrents : Part 4

In the previous articles we designed some scripts to read the IVW log files to work out what the concurrency was, next we will write a script that will try and Estimate what sort of concurrency certain survey criteria might produce. In this article we will take all the information that we have gathered from Data Collection Users to try and produce some example loads.

First off we need to create some variables, from the

 you should be able to work out what each variable is used for.
Dim iNumberOfSurveyDays,iAverageSurveyQuestions,iNumberOfProjects,iPageDuration
Dim iSurveyStarts,iProjectDuration
Dim iPercentage, iHrsInDay, bStartAtSameTime
' How many days do you intend to run surveys for.
' 365 = 1 Year
' 31 = 1 Month
iNumberOfSurveyDays = 14
' Number of Days a project Lasts
iProjectDuration = 14
' Number of hrs in the day
' 24 hrs would be for a global country study that covers all timezones
' 16 hrs would be for a normal day with 8hrs sleep
iHrsInDay = 16
' Do all the surveys start at the same time
' If True then all surveys will start at 10:00 am on day 1
' If False then surveys will start on Random Days during the iNumberOfSurveyDays
bStartAtSameTime = False
' How many questions on average does each questionnaire have.
iAverageSurveyQuestions = 10
' 30 - 60 Seconds per HTML Page Response Rate No Grids.
' 30 - 240 Seconds per HTML Page Response Rate No Grids.
iPageDuration = 120
' How Many Projects will be run on the server in the number of Survey Days above.
iNumberOfProjects = 1
' How many Survey starts do you get. Normaly this is around 30% of your emails
' sent out if no incentives.( Online Only )
iSurveyStarts = 600
' Percentage of Starts in the first start period
' 5 - 10%
iPercentage = 0

Now comes the fun bit, as always you can find the complete

 here. Type in the following
' >>> ------------------------------------- <<<
' THE CODE >>> DO NOT CHANGE ANYTHING PAST THIS LINE <<<
' >>> ------------------------------------- <<<
Dim oFSO, oFile
Dim oStartDate,oRandomDay,oProjectStartDate,oNewDay
Dim iRandomDay,iRandomDayNumber,iRandomSecondsNumber,iRandomDuration
Dim iStartsPerProject,iSpike
Dim iCount,iRows,iProject, iSecondsInDay, iSecondsInDuration
Dim iSurveyStartsCount[999],iTotal,iMissing, iDuration
Dim sProjectID,sRespondentId, sInterviewID, sStartTime, sFinishTime, sDuration
Dim sLastLogEntry, sTimeSlot,sLine
If ( bStartAtSameTime = True ) Then
oStartDate = CDATE("01/01/2010 10:00:00 AM")
Else
oStartDate = CDATE("01/01/2010")
End If

First off we declare all our variables. The plan is we create a file that can be used with the previous scripts so we need the oFSO and oFile objects. Next we have some variables to hold days, we obviously need a start date January the 1st, however if the setting bStartAtSameTime is set to true we say all the surveys are going to start at 10am. Next we have some more variables to store dates. These are mainly used to store the random dates that we will create to simulate when a survey starts. Next we have variables used to store different counts that are used, the main one to notice here is the iSurveyStartsCount, this is an array , and is setup to only handle 999 projects. So if your project number is more than 999 you will need to adjust this figure. Finally we have all the variables that we will use to write out our file. These are the same variable names used in the previous scripts. Next type the following

Set oFSO = CreateObject("Scripting.FileSystemObject")
Set oFile = oFSO.CreateTextFile("D:DATASDLogsMRIConcurrents.csv", True)
oFile.WriteLine("""ProjectID""" + mr.Tab + """RespondentId""" + mr.Tab + _
       """InterviewID""" + mr.Tab + """StartTime""" + mr.Tab + """FinishTime""" + _
       mr.Tab + """Duration""" + mr.Tab + """LastLogEntry""" + mr.Tab + """TimeSlot""")
iStartsPerProject = iSurveyStarts / iNumberOfProjects
' Create Random Project completes
for iRows = 1 to iNumberOfProjects
iSurveyStartsCount[iRows] = (Int((iStartsPerProject - 1) * Rnd()) + 1)
Next
for iRows = 1 to iNumberOfProjects
iTotal = iTotal + iSurveyStartsCount[iRows]
Next
iMissing = ( iSurveyStarts - iTotal ) / iNumberOfProjects
iTotal = 0
for iRows = 1 to iNumberOfProjects
iSurveyStartsCount[iRows] = iSurveyStartsCount[iRows] + iMissing
iTotal = iTotal + iSurveyStartsCount[iRows]
Next

The first couple of lines start our output file and create the heading for it , next we work out how many completes we are going to assign to each project. To do this we take the total number of completes wanted and divide that by the number of projects, this gives us the startsperproject. To make it more real we then for each project create a random number that is equal or less than the number required for each project. Then we add all the random numbers up together and see how many surveys we are missing from the actual total required and work out the remaining number we should add to each project. Once we have this figure we then add it to each project count. The result of this , should be that we see different totals of completes for each generated project.

Next type the following

' 86400 seconds in a 24hr day
iSecondsInDay = iHrsInDay * 60 * 60
For iProject = 1 to iNumberOfProjects
debug.Log("Project" + ctext(iProject))
If ( bStartAtSameTime = True ) Then
oProjectStartDate = oStartDate
Else
oProjectStartDate = DateAdd(oStartDate,"d",(Int((iNumberOfSurveyDays - 1) * Rnd()) + 1))
iRandomSecondsNumber = (Int((86400 - 1) * Rnd()) + 1)
oProjectStartDate = DateAdd(oProjectStartDate,"s",iRandomSecondsNumber)
End If
' Create the Spike
' Work out 1st Duration
iRandomDuration = 0
For iCount = 1 to iAverageSurveyQuestions
iRandomDuration = iRandomDuration + (Int((iPageDuration - 1) * Rnd()) + 1)
Next

We have seen in the IVW log files and experience tells us that when you launch a survey you will see more surveys started when you send out your initial email than you will say a day after. Some people when they get the email will almost certainly click on the link straight away , so to replicate this we take the iPercentage figure and work out how many completes we would get in that spike period. So what do we do first , well we work out the number of seconds in the day, now a full day is 86400 but because we may not be running for a full day , for example , if your survey is going to peoples work email address they are more likely to answer the survey between office hrs so the number of hrs in the day , for that type of survey, it might only be something like 6am to 8pm which is 14 hrs.

Once we have the number of seconds we add the seconds to the project start date. So now we know when the first interview is going to start, the next thing we need to do is to work out the duration of the survey. For the Spike we take each question in turn and work out a random duration that it will take the user to answer that question and add it to a total. This total we will use as the duration for each survey that will be in the spike. Add the following to your script.

iSpike = ( iSurveyStartsCount[iProject] / 100 ) * iPercentage
For iRows = 1 to iSpike
iRandomDayNumber = (Int((iProjectDuration - 1) * Rnd()) + 1)
iRandomSecondsNumber = (Int((iRandomDuration - 1) * Rnd()) + 1)
oRandomDay = DateAdd(oProjectStartDate,"s",iRandomSecondsNumber)
sProjectID = "Project" + CTEXT(iProject)
sRespondentId = "sRespondentIds" + ctext(iRows)
sInterviewID = "sInterviewID"
sStartTime = CTEXT(oRandomDay)
sFinishTime = CTEXT(oRandomDay)
' Convert to mins
iDuration = Round(iRandomSecondsNumber/60)
sDuration = CTEXT(iDuration)
sLastLogEntry = "Interview Random"
sTimeSlot = ""
sLine = """" + sProjectID + """" + mr.Tab + """" + sRespondentID + """" + _
       mr.Tab + """" + sInterviewID + """" + mr.Tab + sStartTime + mr.Tab + _
       sFinishTime + mr.Tab + sDuration + mr.Tab + """" + sLastLogEntry + _
       """" + mr.Tab + sTimeSlot
oFile.WriteLine(sLine)
If ( day( oRandomDay ) <> day(dateadd(oRandomDay,"n",iDuration))) Then
oNewDay = dateadd(DateOnly(oRandomDay),"d",1)
sStartTime = CTEXT(oNewDay) + " 12:00:00 AM"
sFinishTime = CTEXT(oNewDay) + " 12:00:00 AM"
sDuration = CTEXT(iDuration - DateDiff(oRandomDay,oNewDay,"n"))
sLine = """" + sProjectID + """" + mr.Tab + """x" + sRespondentID + """" + _
            mr.Tab + """" + sInterviewID + """" + mr.Tab + sStartTime + mr.Tab + _
            sFinishTime + mr.Tab + sDuration + mr.Tab + """RECORD CARRIED OVER""" + _
            mr.Tab + sTimeSlot
oFile.WriteLine(sLine)
End If
Next

This is quite allot of

 to add but it is very simple to understand, the first thing we do is to work out how many of the total survey starts are going to be in the spike and then for each survey start we work out the random start date using the random duration. what we end up with is basically a set of records that all start at around the same time.( the spike ) . Next we work out some names for project and interviewer details and then set the start and end date of the survey to be the same. At this stage the End date is not used so it does not matter that it is the same.

Next we have to work out the duration number for this survey, and to do that we take the Random Seconds number and divide that by 60 to give us a number of minutes. And lastly we create some text for the log entry. Normaly this would be the completion status of the entry e.g. timeout or Completed etc, but I this case we will just record the fact that it was generated randomly.

Once we have all the basic information we write it out to the file and move on. Now the next block of 
 is to take care of any surveys that may start on one day and finish on another. You might think that this is silly, no survey takes more than 24hrs, but this 
 is not what that is about , if we take the instance that a survey might be done in another country then the time that respondent does the survey could spread across 2 days. So the check is if the random day we generated is not equal to the random day we generated plus the current duration then it must be a new day so we adjust the start and end time accordingly and right the line out to the file. Next type in the following,
' Do the Rest
' Random Number of seconds in the available time.
Dim oStartTime, oEndTime,iShrinkHrs
iSecondsInDuration = iSecondsInDay * iProjectDuration
If ( iHrsInDay <> 24 ) Then
iShrinkHrs = (( 24 - iHrsInDay ) / 2 ) * 60
oStartTime = timeonly(DateAdd(cdate("01/01/2010 12:00:00 AM"),"n",iShrinkHrs))
oEndTime = timeonly(DateAdd(cdate("01/01/2010 12:00:00 AM"),"n",-iShrinkHrs))
End If

The next bit of

 deals with the remaining completes, so the first thing we do is to work out the number of seconds that we have available to create a survey in. This means that if we have 7 days to run this survey in we would have ( 7 * 86400 ) which is 604800 seconds. Next we check to see if the number of hrs in the day is not equal to 24. If it is not , lets say it was a survey sent to people at work , then the time period that a survey can be created in is reduced, for example most people will not be at work at midnight , or early morning before 7. So what we do is to work out the difference in hrs and divide them by 2 and add that number to the start and end dates. So if we had a day that was setup to be 16 hrs long then 24 - 16 = 8 and then divided by 2 is 4 , so a survey would not be allowed between the hrs 12pm to 4am and 8pm to 12pm.

The next bit of 
 is where we generate the remaining survey starts,
For iRows = 1 to iSurveyStartsCount[iProject] - iSpike
TryAgain:
iRandomSecondsNumber = (Int((iSecondsInDuration - 1) * Rnd()) + 1)
oRandomDay = DateAdd(oProjectStartDate,"s",iRandomSecondsNumber)
If Not(( TimeOnly(oRandomDay) >= oStartTime ) and ( TimeOnly(oRandomDay) <= oEndTime ) ) Then
Goto TryAgain
End If
iRandomDuration = 0
For iCount = 1 to iAverageSurveyQuestions
iRandomDuration = iRandomDuration + (Int((iPageDuration - 1) * Rnd()) + 1)
Next
' Convert to mins
iRandomDuration = Round(iRandomDuration/60)
sProjectID = "Project" + CTEXT(iProject)
sRespondentId = "sRespondentIdr" + ctext(iRows)
sInterviewID = "sInterviewID"
sStartTime = CTEXT(oRandomDay)
sFinishTime = CTEXT(oRandomDay)
sDuration = CTEXT(iRandomDuration)
sLastLogEntry = "Interview Random"
sTimeSlot = ""
sLine = """" + sProjectID + """" + mr.Tab + """" + sRespondentID + """" + _
        mr.Tab + """" + sInterviewID + """" + mr.Tab + sStartTime + mr.Tab + _
        sFinishTime + mr.Tab + sDuration + mr.Tab + """" + sLastLogEntry + """" + _
        mr.Tab + sTimeSlot
oFile.WriteLine(sLine)
If ( day( oRandomDay ) <> day(dateadd(oRandomDay,"n",iDuration))) Then
oNewDay = dateadd(DateOnly(oRandomDay),"d",1)
sStartTime = CTEXT(oNewDay) + " 12:00:00 AM"
sFinishTime = CTEXT(oNewDay) + " 12:00:00 AM"

sDuration = CTEXT(iDuration - DateDiff(oRandomDay,oNewDay,"n"))
sLine = """" + sProjectID + """" + mr.Tab + """x" + sRespondentID + """" + mr.Tab + """" + sInterviewID + """" + mr.Tab + sStartTime + mr.Tab + sFinishTime + mr.Tab + sDuration + mr.Tab + """RECORD CARRIED OVER""" + mr.Tab + sTimeSlot
oFile.WriteLine(sLine)
End If
Next
Next
oFile.Close()
Set oFSO = Null

As before the first thing we do is to work out the remaining number of surveys to complete and then for each of them we create a start time based on the number of seconds in our allowable duration and we then check that it is in the allowable time frame, if it is not then we jump back a few lines and try that process again until we get a valid number. Once we have a valid start time we create the survey duration in the same way as we did before, taking each question one at a time and creating a random length that the respondent would take to answer the question. When we have the duration for the survey we then start to write the information to the text file. As before if the survey ends on a different day to the day it started then the script takes this into account and makes another entry with the new day and the remaining mins of the survey. We then loop back up the top to repeat the process for each remaining survey start that we need. Once every start has been completed we save the file and close the File System Object.

Now that we have the full script and it is working , we can run it using your expected paramaters and it should give us a rough idea as to how many real concurrents you could expect on your system. This is an image of the data produced by the script with the basic settings.

Basic Settings

In our next article we will talk briefly on how we can use the information to help us work out the number of session engines that you might need in you new system.

The full

 for this article can be found in :
http://CodeCorner.SmaterDimensions.Com @ Make Concurrents

Leave a Comment

%d bloggers like this: