Test methodology
The objective of this document is to show the performance of Video Stream for 10,000 and 50,000 community members.
Since only a very small subset of users are actually active in the system (and less so for communities versus regular team sites), we are using a concurrency ratio of 1 to 100 for video viewing and browsing (i.e. 100 simultaneously users viewing and browsing represents a community of 10,000 members).
Uploading and encoding is much less at 1 to 500. The concurrency ratio for any given installation may be higher or lower. We used Apache JMeter to monitor test cases for performance results. All tests were run independently.
The test cases include:
- Video viewing test for 10,000 Members with high-quality video
- Video viewing test for 50,000 Members with medium-quality video
- Video Center browsing for 10,000 members
- Encoding performance for 50,000 members
- Video upload performance for 50,000 members
Test results
Video viewing test for 10,000 Members with high-quality video
- Test data
- 100 simultaneous users watching 10 minute 1080p videos
- System statistics (Server A with 100MBit network)
- CPU: 34%
- Network: < 15%
- Disk: Busy, Utilization 178 read/writes per second
- Memory: Normal, 52%, Top Process: w3wp (450,636KB)
- Requests
- GetSecurityToken: Average - 719ms, Median 716ms
- OpenSession: Average - 521 ms, Median 551ms
- GetChunk: Average 5560ms, Median 4500ms
- Comments
- No obvious bottle-necks for this number of users.
Video viewing test for 50,000 members with medium-quality video
- Test data
- 500 simultaneous users watching 10 minute video at 1.15 Mb/s (good quality at default player size and decent quality at full screen)
- System statistics (Server A with 100MBit network)
- CPU: 11%
- Network: 97%
- Disk: Busy, Utilization 7 read/writes per second
- Memory: Normal, 50%, Top Process: w3wp (450,636KB)
- Bytes total/sec: 121,246,125 (925 Mbit/sec)
- Requests
- GetSecurityToken: Average - 308ms, Median 295ms, 90% line 665ms
- OpenSession: Average - 1008 ms, Median 1119ms, 90% line 1152ms
- GetChunk: Average 236ms, Median 113ms, 90% line 627ms (this is for a medium quality stream 1.15Mbps)
- Comments
- The bottle-neck in this test was the network bandwidth between the streaming server and the video player running in the user’s browser. The purpose of this test was to determine how large a community could be supported with a 1 GBit network connection. Note:
Due to smooth streaming, the video player automatically adjusts based on network and client availability to maximize the number of users that can watch an uninterrupted video albeit at a lower quality level.
With 11% CPU usage, ramping up to HD would, understandably, require more network bandwidth.
- The bottle-neck in this test was the network bandwidth between the streaming server and the video player running in the user’s browser. The purpose of this test was to determine how large a community could be supported with a 1 GBit network connection.
Video Center browsing for 10,000 members
- Test data
- 100 simultaneous users browsing a list of 1000 videos using various filters (not including video viewing)
- System statistics (Server A with 100MBit network)
- CPU: 76%
- Network: 20%
- Disk: Normal, Utilization 27 read/writes per second
- Memory: 78%
- Page load times
- Video Center main page: <1 second, max 2.5 seconds (90th percentile)
- Pagination on Video Center page: < 1 second, Max 3.2 seconds (90th percentile)
- Filters (for example, from “Most Liked”, “Most Viewed”. “Most Recent”): 1-2 seconds
- Individual Video Page: <3.3 seconds, max 4.5 seconds (90th percentile)
- Comments
- The bottle-neck in this test was the WFE which can be addressed by adding more WFE’s – a very robust scaling strategy in SharePoint.
Encoding performance for 50,000 members
- Test data
- 100 simultaneous users encoding 50 MB videos
- System statistics (Server C with 100 MB network)
- CPU: 89%
- Network: < 15%
- Disk: Idle, Utilization 38 read/writes per second ,
- Memory: Normal 41%
- Comments
- Predictably, the CPU is the bottle-neck which is why we recommend running the encoding service on its own server.
Video upload performance for 50,000 users
- Test data
- 100 simultaneous users uploading 50 MB files
- System statistics (Server A with a 100 MB network connection to the uploader’s browser)
- CPU: 51%
- Memory: 69%, 1917MB Available
- Top Process: w3wp, Private Working Set: 574,812 KB
- HDD IO/sec: 1091 (> 500), Avg. Disk Queue Length. Mean 3, Minimum 1.064, Maximum 9
- Comments
- Since videos are uploaded in “chunks”, this tends to minimize the CPU, memory and network drain.