QUESTION
What do simulation tests reveal about potential system issues, and what steps are taken for improvement?
1:26:26
·
145 sec
Simulation tests reveal potential system issues, including login problems and system performance under load, leading to improvements with IBM's assistance.
- Simulation tests evaluate both educational logistics, like student and teacher access to assignments, and system performance under load.
- Issues such as login problems and system performance under load are identified, with the latter causing significant problems on a specific day.
- The New York City Public Schools are working with IBM to enhance system performance in nonproduction environments by simulating high user activity.
- IBM's cloud product improvements are expected to benefit not only the New York City Public Schools but also other users.
Rita C. Joseph
1:26:26
when you when you created the simulation, right, I know maybe IBM will be able to answer that.
1:26:32
Does it ever bring red flag?
1:26:34
Do you have a does it give you data that maybe there is a problem, and how do you come in and adjust and fix it?
Scott Strickland
1:26:41
Yep.
1:26:41
And So simulation testing has 2 flavors.
1:26:46
Right.
1:26:47
When we when we maybe simulated a remote day with our students and teachers, we were looking to see Did the students know their username and password?
1:26:54
Could they get to assignment?
1:26:55
Could a teacher create the assignment?
1:26:57
And maybe see how many kids actually logged in and responded that they got to the classroom where actually did the work.
1:27:03
So that's a simulated remote day from a educational standpoint.
1:27:07
The other thing is did our systems get tested in a manner that would recognize that they function well, they didn't give errors, and they could handle load.
1:27:15
And it's that last one that obviously was a problem on the morning of 13th.
1:27:20
So that work is done through what we call automated performance testing, where you work at a nonproduction environment and push it to the limit.
1:27:28
And at some point, this this will keep performing well as you push higher and higher.
1:27:33
And at some point, it starts to either slow down max out its use of its CPU or its storage and it either stops or has long response times to get to an answer.
1:27:44
That second piece is what we're working on with IBM now to say, at if you provide a certain level of TPS, how much throughput can you get?
1:27:53
How many users can come in?
1:27:54
How many successful logins can you get?
1:27:56
And then if you ratchet that capacity output, can you get And how far can you push till you get to the point where we got to on 13th where things didn't get to a successful result by every simulated user that was coming into the system.
Rita C. Joseph
1:28:09
What's what's the timeline for all of that to happen?
Scott Strickland
1:28:12
So we've been working every day since the 13th IBM has got this is a product they offer in the cloud.
1:28:19
So they're working on improving the performance of that product, which will obviously service and other users of this product.
1:28:26
We have our own test environment where we can simulate more things.
1:28:30
We can actually see a user going to teach up, oh, you haven't logged in yet, you get interrupted, get your username and password.
1:28:36
Now you go to teach up, then you go to Google, and we're making sure that all those handoffs work and that the way our system interacts in a real user flow way gets a user from a dead start into their remote learning environment where they can do their their instruction.