LIGHT

  • News
  • Docs
  • Community
  • Reddit
  • GitHub
Star

Regex Replace

Often you need to replace something in your projects across multiple repositories or even multiple organizations or git servers. The manual process would be:

  • Clone all repositories into a workspace
  • Do a full-text search and replace for the workspace
  • Check in all repositories

If you have several repositories, it will not take a long time to do so. However, if you have dozens or hundreds of repositories, it would be a daunting job.

What if you have to do it on a regular basis, for example, if you had to upgrade a library to a certain version across all repositories?

light-bot regex-replace is a task created to do the replacement automatically for us. In this tutorial, we will upgrade Jackson jackson-databind from 2.9.1 to 2.9.4 as 2.9.1 has CVE-2017-17485 deserialization flaw found by one of our customers during a security scan.

Build light-bot

The first step is to build light-bot locally so that we can use the cli tool. Please refer to build light-bot tutorial for details.

Check out light-config-test

Once light-bot is built successfully, you can clone light-config-test repository to the same workspace as light-bot. This repository contains configuration files for the regex-replace task.

Once light-bot is built successfully, you can clone light-config-test repository to the same workspace as light-bot. This repository contains configuration files for the regex-replace task and script to execute the task.

regex-replace configuration file can be found in light-config-test/light-bot/regex-replace/replace-all/config folder. Here is the current content.

regex-replace.yml

# Workspace that is used for this operation. Most of time, this is done on local user home.
workspace: regexreplace
# indicate if you want to skip checkout. yes if you know that all repositories are just checkout
skip_checkout: false
# indicate if you want to skip replace. yes if you have just run the replace first and this time you just want to checkin.
skip_replace: false
# indicate if you want to skip checkin. If you are not comfortable to checkin directly, skip this step and then you can
# skip checkout and replace in second round and only make skip checkin false.
skip_checkin: true
# clone and switch to develop branch or checkout and pull from develop branch. This is to ensure that develop branch for
# each repository is up-to-date for manipulations. Please update skip_checkout to true if you want to bypass this step.
checkout:
  branch: develop
  repository:
  # repositories from github.com  
  - [email protected]:networknt/light-4j.git
  - [email protected]:networknt/openapi-parser.git
  - [email protected]:networknt/light-rest-4j.git
  - [email protected]:networknt/light-graphql-4j.git
  - [email protected]:networknt/light-hybrid-4j.git
  - [email protected]:networknt/light-codegen.git
  - [email protected]:networknt/light-eventuate-4j.git
  - [email protected]:networknt/light-tram-4j.git
  - [email protected]:networknt/light-saga-4j.git
  - [email protected]:networknt/light-session-4j.git
  - [email protected]:networknt/light-proxy.git
  - [email protected]:networknt/light-router.git
  - [email protected]:networknt/light-oauth2.git
  - [email protected]:networknt/light-portal.git
  - [email protected]:networknt/light-example-4j.git
  - [email protected]:networknt/light-docker.git
  - [email protected]:networknt/light-doc.git
  - [email protected]:networknt/light-config-test.git
  - [email protected]:networknt/light-bot.git
  - [email protected]:networknt/light-config-server.git
  - [email protected]:networknt/microservices-framework-benchmark.git
  - [email protected]:networknt/model-config.git
  - [email protected]:networknt/light-portal.git
  - [email protected]:networknt/react-schema-form.git
  - [email protected]:networknt/light-workflow-4j.git
  - [email protected]:networknt/light.git
  - [email protected]:networknt/swagger-bundler.git
  - [email protected]:networknt/http2client-benchmark.git
  - [email protected]:networknt/json-schema-validator-perftest.git
  - [email protected]:networknt/microbenchmark.git
  - [email protected]:networknt/react-schema-form-rc-select.git
  - [email protected]:networknt/light-config-prod.git
  - [email protected]:networknt/react-file-manager.git
  - [email protected]:networknt/light-commerce.git
  - [email protected]:networknt/light-cms.git
  # repositories from lightapi.net
  - [email protected]:lightapi/tokenization.git

# regex replacement from old value to new value during a full text search on certain patterns
# of file names.
replace:
# matched path based on Glob (https://javapapers.com/java/glob-with-java-nio/)
# - glob: "{**/pom.xml,**/pom.xml.rocker.raw}"
#   match: <version.json-schema-validator>\d*\.\d*\.\d*</version.json-schema-validator>
#   old_value: 0.1.10
#   new_value: 0.1.15

- glob: "{**/pom.xml,**/pom.xml.rocker.raw}"
  match: <version.jackson>\d*\.\d*\.\d*</version.jackson>
  old_value: 2.9.1
  new_value: 2.9.4

Customize Config

In the above regex-replace.yml file, you can skip each step. For example, skip the automatic checkin so that you can verify the result and then run the task again with skip checkout and skip replace.

You can change the repositories to your and decide which branch you want to perform the task.

At the end of the file, you can see we have set up to replace jackson version from 2.9.1 to 2.9.4 in all pom.xml and pom.xml.rocker.raw template files in light-codegen

Execute Task

Sometimes you might want to clean up your existing workspace so that you can clone all repositories freshly. It is not necessary but if you want to do it, use the following command.

cd ~
rm -rf regexreplace

To execute the task:

./run.sh

After the first run, you can use the following command to check if the 2.9.1 still exists in the workspace.

cd ~/regexreplace
grep -R --include="pom.xml" "2.9.1" .

You can also, use “git status” to check how many files are updated.

If you are comfortable at the moment, you can update the regex-replace.yml to set skip_checkin to false and rerun the run.sh script. This time, all the repositories will be checked into the remote git server(s).

After you use the tool several times, you will feel comfortable to run the task only once with checkin enabled in the first run.

  • About Light
    • Overview
    • Testimonials
    • What is Light
    • Features
    • Principles
    • Benefits
    • Roadmap
    • Community
    • Articles
    • Videos
    • License
    • Why Light Platform
  • Getting Started
    • Get Started Overview
    • Environment
    • Light Codegen Tool
    • Light Rest 4j
    • Light Tram 4j
    • Light Graphql 4j
    • Light Hybrid 4j
    • Light Eventuate 4j
    • Light Oauth2
    • Light Portal Service
    • Light Proxy Server
    • Light Router Server
    • Light Config Server
    • Light Saga 4j
    • Light Session 4j
    • Webserver
    • Websocket
    • Spring Boot Servlet
  • Architecture
    • Architecture Overview
    • API Category
    • API Gateway
    • Architecture Patterns
    • CQRS
    • Eco System
    • Event Sourcing
    • Fail Fast vs Fail Slow
    • Integration Patterns
    • JavaEE declining
    • Key Distribution
    • Microservices Architecture
    • Microservices Monitoring
    • Microservices Security
    • Microservices Traceability
    • Modular Monolith
    • Platform Ecosystem
    • Plugin Architecture
    • Scalability and Performance
    • Serverless
    • Service Collaboration
    • Service Mesh
    • SOA
    • Spring is bloated
    • Stages of API Adoption
    • Transaction Management
    • Microservices Cross-cutting Concerns Options
    • Service Mesh Plus
    • Service Discovery
  • Design
    • Design Overview
    • Design First vs Code First
    • Desgin Pattern
    • Service Evolution
    • Consumer Contract and Consumer Driven Contract
    • Handling Partial Failure
    • Idempotency
    • Server Life Cycle
    • Environment Segregation
    • Database
    • Decomposition Patterns
    • Http2
    • Test Driven
    • Multi-Tenancy
    • Why check token expiration
    • WebServices to Microservices
  • Cross-Cutting Concerns
    • Concerns Overview
  • API Styles
    • Light-4j for absolute performance
    • Style Overview
    • Distributed session on IMDG
    • Hybrid Serverless Modularized Monolithic
    • Kafka - Event Sourcing and CQRS
    • REST - Representational state transfer
    • Web Server with Light
    • Websocket with Light
    • Spring Boot Integration
    • Single Page Application
    • GraphQL - A query language for your API
    • Light IBM MQ
    • Light AWS Lambda
    • Chaos Monkey
  • Infrastructure Services
    • Service Overview
    • Light Proxy
    • Light Mesh
    • Light Router
    • Light Portal
    • Messaging Infrastructure
    • Centralized Logging
    • COVID-19
    • Light OAuth2
    • Metrics and Alerts
    • Config Server
    • Tokenization
    • Light Controller
  • Tool Chain
    • Tool Chain Overview
  • Utility Library
  • Service Consumer
    • Service Consumer
  • Development
    • Development Overview
  • Deployment
    • Deployment Overview
    • Frontend Backend
    • Linux Service
    • Windows Service
    • Install Eventuate on Windows
    • Secure API
    • Client vs light-router
    • Memory Limit
    • Deploy to Kubernetes
  • Benchmark
    • Benchmark Overview
  • Tutorial
    • Tutorial Overview
  • Troubleshooting
    • Troubleshoot
  • FAQ
    • FAQ Overview
  • Milestones
  • Contribute
    • Contribute to Light
    • Development
    • Documentation
    • Example
    • Tutorial
“Regex Replace” was last updated: July 5, 2021: fixes #275 checked and corrected grammar/spelling for majority of pages (#276) (b3bbb7b)
Improve this page
  • News
  • Docs
  • Community
  • Reddit
  • GitHub
  • About Light
  • Getting Started
  • Architecture
  • Design
  • Cross-Cutting Concerns
  • API Styles
  • Infrastructure Services
  • Tool Chain
  • Utility Library
  • Service Consumer
  • Development
  • Deployment
  • Benchmark
  • Tutorial
  • Troubleshooting
  • FAQ
  • Milestones
  • Contribute