Commit 4290424f authored by Daniel Smith's avatar Daniel Smith

Added environment variables for SMTP Server config. Updated Readme. Switched to LF line endings

parent 6829ebb0
...@@ -47,8 +47,10 @@ The framework is platform independent and can be deployed to as many clients as ...@@ -47,8 +47,10 @@ The framework is platform independent and can be deployed to as many clients as
Optional parameters are set via environment variables. The following options are available. Optional parameters are set via environment variables. The following options are available.
1. ```HTTP_PORT``` - The port on which to run the server. All instances of this system on a network must be configured to use the same port. ```Default: 8080``` 1. ```HTTP_PORT``` - The port on which to run the server. All instances of this system on a network must be configured to use the same port. ```Default: 8080```
2. ```BUILD_CORES``` - The number of CPU threads to use when building Qt ```Default: 8``` 2. ```BUILD_CORES``` - The number of CPU threads to use when building Qt ```Default: [All available cores]```
3. ```VS_DEV_ENV``` - The full path to VsDevCmd.bat provided by Microsoft Visual Studio ```Default: C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/Common7/Tools/VsDevCmd.bat``` 3. ```VS_DEV_ENV``` - The full path to VsDevCmd.bat provided by Microsoft Visual Studio ```Default: "C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/Common7/Tools/VsDevCmd.bat"```
4. ```SMTP_SERVER``` - A SMTP server that can send emails. Authentication not supported - The mail server must accept anonymous connections. ```Default: smtp.intra.qt.io```
5. ```SMTP_PORT``` - Specify the port to connect to the mail server. ```Default: 25```
## Using the interface ## Using the interface
...@@ -57,7 +59,11 @@ The framework is platform independent and can be deployed to as many clients as ...@@ -57,7 +59,11 @@ The framework is platform independent and can be deployed to as many clients as
1. If using the bisect or two commit comparison, the commits must be on the same major branch. 1. If using the bisect or two commit comparison, the commits must be on the same major branch.
2. You need to know which module the commit(s) were made in (i.e. qtdeclarative or qtbase for example) 2. You need to know which module the commit(s) were made in (i.e. qtdeclarative or qtbase for example)
3. You need to have a rough idea of the expected performance regression. Testing regressions smaller than ~5% is not recommended due to natural test noise. 3. You need to have a rough idea of the expected performance regression. Testing regressions smaller than ~5% is not recommended due to natural test noise.
4. It is **strongly recommended** that a QMLBench Regression Finder running as a slave is hosted on a machine that is performing no other tasks and has been configured for performance stability.
4. You need to know a specific QMLBench test that is affected by a given commit's changes. 4. You need to know a specific QMLBench test that is affected by a given commit's changes.
- You may also write and upload custom QMLBench benchmarks.
- See [QMLBench](https://github.com/qt-labs/qmlbench) for an overview of the QMLBench test framework.
- The benchmark [creation/quick.item/delegates_item.qml](https://github.com/qt-labs/qmlbench/blob/master/benchmarks/auto/creation/quick.item/delegates_item.qml) may provide a good starting point for writing your own QMLBench benchmark.
### Automatic Bisect ### Automatic Bisect
This mode takes two commits from the same major branch (i.e. 5.12) and emulates a git-bisect like behavior, testing various commits in-between the known good and known bad commits to find a performance regression. This mode takes two commits from the same major branch (i.e. 5.12) and emulates a git-bisect like behavior, testing various commits in-between the known good and known bad commits to find a performance regression.
...@@ -74,3 +80,17 @@ The framework is platform independent and can be deployed to as many clients as ...@@ -74,3 +80,17 @@ The framework is platform independent and can be deployed to as many clients as
Notes: Notes:
1. Though the commits can be WIP or unmerged, they must still be in the same module and of the same branch. Use multiple runs of the single-commit test mode to test commits in different branches or modules. 1. Though the commits can be WIP or unmerged, they must still be in the same module and of the same branch. Use multiple runs of the single-commit test mode to test commits in different branches or modules.
## Troubleshooting
### Common problems
1. **Build failures**
1. Bad checkout of some qt repo. If a test fails to build when you know it should, try simply deleting the QtBuild directory on the slave and re-stage the rest run.
2. **Failure to produce a test result**
1. There is no validation of custom QML benchmark file. If everything looks good in the build but QMLBench fails to produce a result or crashes, make sure your custom QMLBench benchmark .qml file is formatted correctly.
2. OpenGL is currently required on all platforms. If no OpenGL is installed, or if Angle is selected on windows but unsupported on the host, QMLBench will likely crash.
3. **Test result instability**
1. If you're using icecc in your linux environments and encounter test performance instability, ensure icecc's config is set to disable remote jobs.
2. Be sure the system was not attempting to perform automatic updates.
3. Increase the count of test repeats. Default is 10. Each repeat runs for approximately 20 seconds.
- Check regressionFinder.py::runBenchmark() to adjust the number of test repeats.
...@@ -47,6 +47,8 @@ var webPort = 8080; ...@@ -47,6 +47,8 @@ var webPort = 8080;
var buildCores; var buildCores;
var vsDevEnv; var vsDevEnv;
var testNames = []; var testNames = [];
var smtpServer = 'smtp.intra.qt.io';
var smtpPort = 25;
// Initialize some things and check our startup parameters // Initialize some things and check our startup parameters
...@@ -109,7 +111,12 @@ if (process.env.BUILD_CORES){ ...@@ -109,7 +111,12 @@ if (process.env.BUILD_CORES){
if (process.env.VS_DEV_ENV){ if (process.env.VS_DEV_ENV){
vsDevEnv = process.env.VS_DEV_ENV; vsDevEnv = process.env.VS_DEV_ENV;
} }
if (process.env.SMTP_SERVER){
smtpServer = process.env.SMTP_SERVER;
}
if (process.env.SMTP_SERVER){
smtpPort = Number(process.env.SMTP_PORT);
}
notifyJob.on('updateRemote', function(recursive){ notifyJob.on('updateRemote', function(recursive){
// This signal should be emitted when an update from a remote worker is received. // This signal should be emitted when an update from a remote worker is received.
if (!updateLockout){ if (!updateLockout){
...@@ -313,9 +320,9 @@ notifyJob.on('JobDone', function(){ ...@@ -313,9 +320,9 @@ notifyJob.on('JobDone', function(){
// create reusable transporter object using the default SMTP transport // create reusable transporter object using the default SMTP transport
let transporter = nodemailer.createTransport({ let transporter = nodemailer.createTransport({
host: 'smtp.intra.qt.io', host: smtpServer,
port: 25, port: smtpPort,
secure: false, // true for 465, false for other ports secure: smtpPort === 465 ? true : false, // true for 465, false for other ports
tls: { tls: {
rejectUnauthorized: false, rejectUnauthorized: false,
}, },
...@@ -543,7 +550,7 @@ function errorsPage(req, res){ ...@@ -543,7 +550,7 @@ function errorsPage(req, res){
`; `;
} }
} else { } else {
pageHTML += ` pageHTML = pageHTML + `
<h2>💤 Things are looking pretty quiet around here... 💤</h2> <h2>💤 Things are looking pretty quiet around here... 💤</h2>
`; `;
} }
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment