4.5. CI/CD Static Analysis

4.5.1. `SonarLint`

https://www.sonarlint.org
Plugin do IDE

4.5.2. `SonarScanner`

4.5.3. `SonarQube`

SonarQube software (previously called Sonar) is an open source quality management platform, dedicated to continuously analyze and measure technical quality, from project portfolio to method.

Learn more at:

https://sonarqube.com

http://docs.sonarqube.org/display/SONAR/Documentation

https://sonarqube.com/dashboard/index?did=143

https://sonarqube.com/governance?id=662857

4.5.4. Przygotowanie środowiska statycznej analizy

Uruchomienie:

$ cd PROJECT_DIRECTORY
$ docker run --rm -d --name sonarqube -p 9000:9000 -v $(pwd):/src sonarqube
$ docker exec -u 0 -it sonarqube bash

    curl -sL https://deb.nodesource.com/setup_8.x -o /opt/node.sh
    bash /opt/node.sh
    apt install -y nodejs
    wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-3.3.0.1492.zip -O /opt/sonar-scanner.zip
    unzip -d /opt/ /opt/sonar-scanner.zip
    ln -s /opt/sonar-scanner-*/bin/sonar-scanner /usr/bin/sonar-scanner
    VERSION=$(cd /src/ && hg log -l 1 --template '{node}\n')

    apt install -y python-pip pylint python-coverage python-nose
    pip install -r /src/requirements.txt

Konfiguracja:

Quality Profile -> Python

Skopiuj profil "Sonar way" i nazwij nowy jako "PyLint"

Trybik (prawy górny róg) -> Activate more rules

Przefiltruj listę (lewy dolny róg) po "Repository" równym "PyLint"

Bulk Change (góra ekrany) -> Activate in "PyLint" -> zaakceptuj

Ustaw "PyLint jako domyślny"

Uruchom analizę

Administration -> Marketplace
- Zainstalować plugin HTML

4.5.5. Python

Code 4.5. SonarScanner config for static analysis of Python code

## Sonar Server
sonar.host.url=http://localhost:9000/
sonar.login=admin
sonar.password=admin

## About Project
sonar.projectKey=python
sonar.projectName=Python
sonar.projectDescription=
sonar.links.homepage=
sonar.links.scm=
sonar.links.issue=
sonar.links.ci=

## Analysis
sonar.language=py
sonar.projectBaseDir=/src/
sonar.sources=.
sonar.sourceEncoding=UTF-8
#sonar.scm.provider=hg

## Output
sonar.verbose=false
sonar.log.level=INFO
sonar.showProfiling=false
#sonar.scanner.dumpToFile=/tmp/sonar-python.properties

## Python
sonar.inclusions=**/*.py
sonar.exclusions=**/migrations/**,**/*.pyc,**/__pycache__/**
sonar.python.pylint=/usr/bin/pylint
sonar.python.pylint_config=.pylintrc
sonar.python.xunit.skipDetails=false
sonar.python.xunit.reportPath=xunit.xml
sonar.python.coverage.reportPath=coverage.xml
sonar.core.codeCoveragePlugin=cobertura

## Turn off these rules
## python:s100: "Method names should comply with a naming convention"
## gives many false positives when overriding
## TestCase methods (such as setUp and tearDown) in test files
sonar.issue.ignore.multicriteria=e1,e2
sonar.issue.ignore.multicriteria.e1.ruleKey=python:S100
sonar.issue.ignore.multicriteria.e1.resourceKey=**/tests.py
sonar.issue.ignore.multicriteria.e2.ruleKey=python:S100
sonar.issue.ignore.multicriteria.e2.resourceKey=**/tests.py

4.5.6. CSS

Code 4.6. SonarScanner config for static analysis of CSS code

## Sonar Server
sonar.host.url=http://localhost:9000/
sonar.login=admin
sonar.password=admin

## About Project
sonar.projectKey=css
sonar.projectName=CSS
sonar.projectDescription=
sonar.links.homepage=
sonar.links.scm=
sonar.links.issue=
sonar.links.ci=

## Analysis
sonar.language=css
sonar.projectBaseDir=/src/
sonar.sources=.
sonar.sourceEncoding=UTF-8
sonar.scm.provider=git

## Output
sonar.verbose=false
sonar.log.level=INFO
sonar.showProfiling=false
#sonar.scanner.dumpToFile=/tmp/sonar-css.properties

## CSS
sonar.inclusions=**/*.css,**/*.less,**/*.scss
sonar.exclusions=**/tinymce.**,**/jquery.*
sonar.css.node=/usr/bin/node
sonar.css.file.suffixes=.css,.less,.scss

4.5.7. JavaScript

Code 4.7. SonarScanner config for static analysis of JavaScript code

## Sonar Server
sonar.host.url=http://localhost:9000/
sonar.login=admin
sonar.password=admin

## About Project
sonar.projectKey=javascript
sonar.projectName=JavaScript
sonar.projectDescription=
sonar.links.homepage=
sonar.links.scm=
sonar.links.issue=
sonar.links.ci=

## Analysis
sonar.language=js
sonar.projectBaseDir=/src/
sonar.sources=.
sonar.sourceEncoding=UTF-8
sonar.scm.provider=git

## Output
sonar.verbose=false
sonar.log.level=INFO
sonar.showProfiling=false
#sonar.scanner.dumpToFile=/tmp/sonar-javascript.properties

## JavaScript
sonar.inclusions=**/*.js,**/*.jsx,**/*.vue
sonar.exclusions=**/tinymce.**,**/jquery.*
sonar.javascript.jQueryObjectAliases=$,jQuery
sonar.javascript.environments=amd,applescript,atomtest,browser,commonjs,couch,embertest,greasemonkey,jasmine,jest,jquery,meteor,mocha,mongo,nashorn,node,phantomjs,prototypejs,protractor,qunit,rhino,serviceworker,shared-node-browser,shelljs,webextensions,worker,wsh,yui
sonar.javascript.globals=angular,goog,google,OpenLayers,d3,dojo,dojox,dijit,Backbone,moment,casper
sonar.javascript.exclusions=**/node_modules/**,**/bower_components/**
sonar.nodejs.executable=/usr/bin/node

4.5.8. Multi-language

Code 4.8. SonarScanner config for static analysis of Multilanguage code

## Sonar Server
sonar.host.url=http://localhost:9000/
sonar.login=admin
sonar.password=admin

## About Project
sonar.projectKey=multilanguage
sonar.projectName=Multi-language
sonar.projectDescription=
sonar.links.homepage=
sonar.links.scm=
sonar.links.issue=
sonar.links.ci=

## Analysis
#sonar.language=
sonar.projectBaseDir=/src/
sonar.sources=.
sonar.sourceEncoding=UTF-8
sonar.scm.provider=git

## Output
sonar.verbose=false
sonar.log.level=INFO
sonar.showProfiling=false
#sonar.scanner.dumpToFile=/tmp/sonar-multilanguage.properties

sonar.inclusions=**/*.css,**/*.less,**/*.scss,**/*.html,**/*.xhtml,**/*.jspf,**/*.jspx,**/*.cshtml,**/*.vbhtml,**/*.aspx,**/*.ascx,**/*.rhtml,**/*.erb,**/*.shtm,**/*.shtml,**/*.js,**/*.jsx,**/*.vue,**/*.py
sonar.exclusions=**/tinymce.**,**/jquery.*,**/sitemap.xml,**/migrations/**,**/*.pyc,**/__pycache__/**

## CSS
sonar.css.node=/usr/bin/node
sonar.css.file.suffixes=.css,.less,.scss

## JavaScript
sonar.javascript.jQueryObjectAliases=$,jQuery
sonar.javascript.environments=amd,applescript,atomtest,browser,commonjs,couch,embertest,greasemonkey,jasmine,jest,jquery,meteor,mocha,mongo,nashorn,node,phantomjs,prototypejs,protractor,qunit,rhino,serviceworker,shared-node-browser,shelljs,webextensions,worker,wsh,yui
sonar.javascript.globals=angular,goog,google,OpenLayers,d3,dojo,dojox,dijit,Backbone,moment,casper
sonar.javascript.exclusions=**/node_modules/**,**/bower_components/**
sonar.nodejs.executable=/usr/bin/node

## Python
sonar.python.pylint=/usr/bin/pylint
sonar.python.pylint_config=.pylintrc
sonar.python.xunit.skipDetails=false
sonar.python.xunit.reportPath=xunit.xml
sonar.python.coverage.reportPath=coverage.xml
sonar.core.codeCoveragePlugin=cobertura

## Turn off these rules
## python:s100: "Method names should comply with a naming convention"
## gives many false positives when overriding
## TestCase methods (such as setUp and tearDown) in test files
sonar.issue.ignore.multicriteria=e1,e2
sonar.issue.ignore.multicriteria.e1.ruleKey=python:S100
sonar.issue.ignore.multicriteria.e1.resourceKey=**/tests.py
sonar.issue.ignore.multicriteria.e2.ruleKey=python:S100
sonar.issue.ignore.multicriteria.e2.resourceKey=**/tests.py

4.5.9. `Coverage`

About:

Coverage.py measures code coverage, typically during test execution. It uses the code analysis tools and tracing hooks provided in the Python standard library to determine which lines are executable, and which have been executed.

Install:

$ pip install coverage

Usage:

$ coverage run FILE.py
$ coverage report -m
$ nosetests --with-coverage --cover-erase --cover-xml --cover-inclusive --with-xunit --xunit-file=xunit.xml --cover-xml-file=coverage.xml

Use coverage run to run your program and gather data:

$ coverage run myprogram.py arg1 arg2
blah blah ..your program's output.. blah blah

Use coverage report to report on the results:

$ coverage report -m
Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
myprogram.py                 20      4    80%   33-35, 39
mymodule.py                  56      6    89%   17-23
-------------------------------------------------------
TOTAL                        76     10    87%

For a nicer presentation, use coverage html to get annotated HTML listings detailing missed lines:

$ coverage html

Larn more at:

https://pypi.python.org/pypi/coverage

https://coverage.readthedocs.io/

4.5.10. `pylama`

$ pylama --linters pylint --skip='*/migrations/*' --abspath /src

Code 4.9. setup.cfg

[pylama]
format = pylint
skip = */.tox/*,*/.env/*
linters = pylint,mccabe,eradicate,isort,pyflakes,pydocstyle,pycodestyle
ignore = F0401,C0111,E731

[pylama:pycodestyle]
max_line_length = 300

[pylama:pyflakes]
builtins = _

[pylama:pylint]
max_line_length = 100
disable = R

[pylama:*/pylama/main.py]
ignore = C901,R0914,W0212
select = R

[pylama:*/tests.py]
ignore = C0110

[pylama:*/setup.py]
skip = 1

4.5.11. `bandit`

Sprawdzanie kodu pod kątem podatności bezpieczeństwa

$ pip install bandit
$ bandit --recursive /src/

4.5.12. `pycodestyle`

$ pip install pycodestyle
$ pycodestyle --max-line-length=79 --exclude=*/migrations/* .

4.5.13. `safety`

$ pip install safety
$ safety check -r /src/requirements.txt

4.5. CI/CD Static Analysis

4.5.1. SonarLint

4.5.2. SonarScanner

4.5.3. SonarQube