How to contribute to Taranta **************************** The collaboration started between software teams in MaxIV and the Square Kilometer Array Observatory (SKAO) to jointly work on developing the Taranta (formerly Webjive) suite and it is now open to the whole TANGO Community. Taranta is the name of a family of products that can be used to provide a web-based user interface to a Tango Control System. How to contribute? ================== - Join the #taranta-webjive slack channel in the Tango Slack Domain (http://tango-controls.slack.com/) - Attend the Taranta Weekly meeting, every Friday from 9:00 AM to 10:00 AM UTC on Google Meet (in the slack channel you will receive the details) - Add feedback, new desiderata or report bugs in our GitLab Repository, in the Issue Section - Contribute to code and raise your merge request - If you want to contribute with a merge request and you are not among the members of `taranta repos `, send a request to our `slack channel `_. before forking the project. What are the Products involved in the Taranta Suite collaboration? ================================================================== .. list-table:: :widths: 1 1 1 1 :header-rows: 1 * - Repository - Docs - Purpose - Description * - https://gitlab.com/tango-controls/web/taranta.git - |Taranta_Docs| - A tool for creating Dashboards for interacting with the devices within a Tango Control System - The tool for developing these dashboards is taranta itself. * - https://gitlab.com/tango-controls/web/tangogql.git - |TangoGQL_Docs| - Queryable access to a Tango Control System that can be used by the dashboard creation tool - Currently, this is a TangoGQL. A GraphQL web server that integrates with the TangoDB services and cancommunicate directly with tango devices. * - https://gitlab.com/tango-controls/web/taranta-dashboard.git - |Taranta Dashboard_Docs| - Storing and Sharing the configuration of developed dashboards between users - A MongoDB based dashboard repository for storing taranta dashboard layouts * - https://gitlab.com/tango-controls/web/taranta-auth.git - |Taranta Auth_Docs| - Authorization and Authentication for the tools - A simple authentication and authorization service for the taranta and TangoGQL tools that can be hooked into a corporate authentication solution * - https://gitlab.com/tango-controls/web/taranta-suite.git - - - Supporting further development - A set of developer scripts and tools used for setting up and developing these related products. Taranta Roadmap =============== Taranta development follows a roadmap, defined and maintained by MaxIV and SKA institutes and shared with the community. MaxIV and SKA adopt an agile software development approach; in particular SKA is using SAFe and therefore code is being produced with two different cadences: a 3-month cadence organized in 2-week sprints. For this reason the roadmap is structured into plans covering different horizons. The **short-term** plan is a 3-months plan and includes the functionalities and improvements that are identified by collecting feedback from SKA and MaxIV operators and Tango Community and are agreed upon between core developers based on priorities. The short-term plan represents the current Taranta development and is pretty well defined in terms of expected deliverables and timeline The **medium-term** and the **long-term plan** are a collection of desired features and functionalities not yet designed and scheduled. Usually, the medium-term plan includes features that are sized and prioritized, but not scheduled and probably will be developed in the next 6-9 months. Instead, the long-term plan includes those that are not sized either prioritized. Every three months, at the end of the short-term plan, the teams schedule, size, and prioritize the items in the medium and long term plans and discuss also the new features to develop in the next short-term plan. Prioritization process ====================== Priorities are agreed on a regular basis between the institutes' users representatives The outcomes of the SKA planning and joint discussions with Max IV are: - An agreed set of common priorities between SKA and MaxIV. - An updated backlog of Taranta suite features recorded as GitLab Issues against the relevant project ( general issues being raised against webjive-develop ) - A single view of all the GitLab issues across the Taranta suite that summarises the current GitLab Issues from each of the contributing projects. A plan for the next 3 months of work covering which features are going to be worked on in which 2wk sprint, with these features having an approximate size, acceptance criteria, an allocated feature owner and agreement on which team is best placed to tackle them. (It is important that SKA discuss their priorities with MaxIV and vice versa before tickets are raised) Once accepted as part of a team’s work for the next period the GitLab issue should be updated to include a cross-reference to the internally tracked work item (JIRA for SKA / TAGIA for MaxIV) Development Process =================== Taranta repository contains different branches. The two main branches are: - **master** branch that should always be deployable and usable and contains the stable version. - **develop** branch, supported by Continuous Integration by the community, and represents the candidate version. Changes are made via short-lived feature branches that are merged back into develop. Other branches are composed of code under development that, after a review, will be merged into the develop branch. Every development has to start creating a new branch from the develop branch. An external repository is preferred. After adding the new changes containing also the tests and documentation, it is possible to create a Merge Request to the develop branch in the MaxIV repository. Once accepted by the reviewers, it is possible to merge the code, resolving possible conflicts. Make sure that the tests pass and cover at least 75% of the new code and the documentation is in place before opening a new merge request. It is possible to keep the MR on DRAFT in the meanwhile. Merge requests without tests and documentation will be refused. Stable version ============== Every three months a new version is released if enough additions are developed. Based on its contents and backwards compatibility issues, it would be a major/minor number bump. If something needs updates because it breaks backwards compatibility, e.g. some widget needs extra definitions, a script should be provided. Compatibility issues should be raised and notified as soon as they are detected, so each party can plan and react to their local updates. Once the master branch is updated with the stable version coming from the develop branch, it is guaranteed the release is stable within the visibility of 6 months. Every next update will not break compatibility for **at least 6 month**. In case a candidate release is identified in the develop branch which breaks the compatibility, it is guaranteed that it will not merge into the master within 6 months from the last stable release. In case a bug is fixed in the develop branch, the same patch will be added also in the master without breaking the code. \ |IMG1|\ Versioning convention ===================== \ |IMG2|\ - Update patch - Commit fixes, update only on documentation changes - Update minor - Add new functionality (ex new widget) that are retro-compatible with previous dashboards/version (documentation is updated) - Update major - Only on new breaks backward compatibility - Use of suffix - s -> stable - c -> candidate Making Changes ============== - **HotFix/Emergency** Fixes are reviewed quickly by a second developer on the same side and communicated retrospectively - **Review policy for planned changes** when a change is ready for merging with the “develop” branch the party responsible raises a merge request, requesting a review with a priority (urgent, high medium, low priority). - If it is a minor change related to a feature recently merged, it is possible to work directly on the develop branch. - A priority defines generally how urgently the review should be actioned. - In the **MaxIV-SKA** collaboration, this is on a best efforts basis without any specific deadlines. If it is not possible for the other party to review the code then the code will be merged back to the develop after peer review by another developer within the collaboration and after **2 working days**. - For external collaborations, the review proceeds on a best effort basis without any specific deadlines Testing ======= The tests for these projects are not currently comprehensive in terms of coverage. The intention is to improve this over time. We want to ensure that new code developed is delivered with tests that demonstrate that it is fit for purpose, and is documented in a way that makes it easy to maintain. At the moment we rely on automated unit and integration tests and on manually executed end-to-end tests. Manual tests will be progressively based on exploratory testing. Testing legacy code ------------------- - It is **not necessary to add tests on existing code although being very welcome**, but is it **required** if the codebase is changed. - When a **bug in legacy code has been discovered**: open a GitLab issue, create a test that reveals the bug, and then fix the bug. In this way, it is possible to improve test code-coverage. Testing of new code or changed functionality: --------------------------------------------- - **All new or changed code should have tests** - **All new or changed code should have user-related documentation** - **The “develop” branch should always be clean and deployable** - no breaking code, all tests and linting OK. The CI/CD should run cleanly on both sides. Test coverage ------------- Test coverage for new code and changed functionality should be at least 75%. It will not be accepted merge requests that don't respect the minimum code coverage limit. Test tools ------------- The test tools used to test the Taranta suite divided in two parts: - Javascript tools, to test the javascript-based project: Taranta, Taranta-auth, dashboard-repo - Python tools, to test TangoGQL, which is python-based Javascript tools are: - jest: a JavaScript testing framework with a focus on simplicity and support for large web applications. It works with projects using Babel, TypeScript, Node.js, React, Angular, Vue.js and Svelte. - RTL: React Testing Library is a testing utility for React that encourages testing user interactions and behavior rather than implementation details. - enzyme (https://enzymejs.github.io/enzyme/): a JavaScript Testing utility for React that makes it easier to test React Components' output. It is also possible to manipulate, traverse, and in some ways simulate runtime given the output. It is used to test Taranta, which is React-based .. warning:: However, please note that starting from Taranta version 2.8.1 with the introduction of React 18, enzyme has been deprecated and may not be supported in future versions. It is recommended to use React Testing Library (RTL). In Taranta, the existing tests using enzyme are maintained through compatibility libraries and will be translated with the new libraries. Following the React recommendation (https://reactjs.org/docs/faq-structure.html), test file should be in the same directory as the file to test, using the same name with .test before the file extension, as the following example: :: components/ └── NavBar/ ├── NavBar.tsx └── NavBar.test.tsx Once created the test, it is possible to run the test using the command: .. code-block:: bash npm test It creates a report in the /coverage folder of the project, containing the result of the test run. ``npm test`` script runs with a parameter ``--runInBand``. This parameter runs all tests serially in the current process, rather than creating a worker pool of child processes that run tests, but it may cause slowdowns. You can use the script .. code-block:: bash npm run test-fast or .. code-block:: bash npm run test-fast:coverage to run tests faster by creating a worker pool of child processes, but some tests containing ``waitFor``, ``await``, ``fireEvent``, may fail if the machine does not have sufficient resources. Therefore, we recommend using these scripts with caution or running the same ``npm test`` if any tests fail to ensure that it is not due to local resources. Python tool used to test TangoGQL is pytest (https://docs.pytest.org/en/stable/) Test & pipeline --------------- Whenever a commit is made, a pipeline is triggered to automatically run tests. It is mandatory for the pipeline to pass successfully before opening a new Merge Request; otherwise, it won't be approved. The GitLab pipeline, in addition to running tests automatically, generates the test coverage report. You can view the reports by first accessing the pipeline from the GitLab menu following the path ``Build -> Pipeline``. Once you've located your pipeline, click on it to open the corresponding test job. On the right pane, click on the ``Browse`` tab. This will open the artifact folder. Navigate to ``build -> reports`` and click on the ``index.html`` file to view the complete test report. Documentation ============= Each code change, whether it's a refactor or a new feature, has to include user documentation. It could be a new document or an update of the existing one. Each merge request has to contain also the new feature/updated feature/bug fixing in the changelog (https://webjive.readthedocs.io/en/latest/history.html), in the WIP section. The documentation is contained in the docs folder and it is based on ReadTheDocs (https://readthedocs.org/). The process to add new documentation is: - create or update the existing documentation in the docs folder - add the link, if it is a new document, in the index.rst - locally compile the documentation When the branch is merged back into the develop branch, an automatic procedure re-builds the documentation and publishes the new one. Coding Standards and Programming Language Conventions ----------------------------------------------------- Taranta and related JavaScript based projects ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We are starting from the principle that we want to move towards a defined coding standard, but without forcing a large amount of cosmetic change on the existing codebase. The following tools are integrated into the environments. This list may be expanded or change as the needs of the project changes. .. list-table:: :widths: 1 2 :header-rows: 1 * - Purpose - Tool of Choice * - linting of files - eslint, @typescript-eslint/parser and @typescript-eslint/eslint-plugin * - testing of code - jest, ts-jest, enzyme, * - formatting of code - prettier * - code coverage - jest * - dependency management - npm * - CI/CD - GitLab pipeline (gitlab-ci.yml file) All existing code should as a minimum conform to the de-facto linting and formatting rules within the workspace. These are currently relaxed in a number of areas, but the plan would be to improve the quality of the code over time enforcing stricter rules based on best practices defined by Airbnb and Microsoft after discussion. Any exceptions to this would be documented on the publicly accessible SKA developer guidance for javascript. For personal linting or formatting of code, it is suggested that developers use the appropriate AirBnB standards rules and plugins for their preferred editor. Guidance for set-up and configuration to be supplied as part of the readme on the projects Use of Typescript ^^^^^^^^^^^^^^^^^ The use of Typescript is acknowledged and supported. It is quite acceptable for TSX files to contain JSX syntax. Typescript code should conform to the current typescript rules for static typing (currently 2.7 is enforced 3.3 is suggested for any new code) For compatibility with the current codebase, the following rules are not enforced however any new or change code should however be written so that it would compile and run with these rules in place - **strictFunctionTypes** Ensure that all functions can be proved to be type safe. [2]_ - **strictPropertyInitialization** Ensures that all properties are initialized for every possible code path. [3]_ - **noImplicitAny** Currently if the compiler cannot infer the variable type based on how it's used it silently defaults the type to any. At some point, we want to switch this to true so that if the TypeScript compiler cannot infer the type, it still generates the JavaScript files, but it also reports an error. This stricter type checking catches more unintentional errors at compile time. Code Structure ^^^^^^^^^^^^^^ The code should, in general, be grouped by features or routes. [4]_ with CSS, JS, and tests grouped together inside folders. Follow the existing taranta structure where possible: * **dashboard** : code relating to the main dashboard display and * **jive** : code relating to the device lists and RHS panel * **shared** code used by both Within this structure, there is a separation between code related to the state management and the widgets presented on the display. The folder structure within ‘components’ reflecting a hierarchical view of the individual components. .. bottom of content .. |IMG1| image:: _static/img/contribute_branch_update.png .. |IMG2| image:: _static/img/contribute_versioning.png