Skip to content

Commit

Permalink
install from pipfile (#1152)
Browse files Browse the repository at this point in the history
  • Loading branch information
SharonHart authored Aug 23, 2023
1 parent fa861aa commit 18090c6
Show file tree
Hide file tree
Showing 18 changed files with 19 additions and 3,945 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ ENV/
env.bak/
venv.bak/
*venv/
*Pipfile.lock

# Spyder project settings
.spyderproject
Expand Down
2 changes: 1 addition & 1 deletion .pipelines/templates/build-analyzer.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ steps:
workingDirectory: 'presidio-analyzer'
script: |
set -eux # fail on error
pipenv sync --dev
pipenv install --deploy --dev
pipenv run python -m spacy download en_core_web_lg
- template: ./build-python.yml
Expand Down
2 changes: 1 addition & 1 deletion .pipelines/templates/build-anonymizer.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ steps:
workingDirectory: 'presidio-anonymizer'
script: |
set -eux # fail on error
pipenv sync --dev
pipenv install --deploy --dev
- template: ./build-python.yml
parameters:
Expand Down
2 changes: 1 addition & 1 deletion .pipelines/templates/build-cli.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ steps:
workingDirectory: 'presidio-cli'
script: |
set -eux # fail on error
pipenv sync --dev
pipenv install --deploy --dev
pipenv run python -m spacy download en_core_web_lg
- template: ./build-python.yml
parameters:
Expand Down
2 changes: 1 addition & 1 deletion .pipelines/templates/build-image-redactor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ steps:
workingDirectory: 'presidio-image-redactor'
script: |
set -eux # fail on error
pipenv sync --dev
pipenv install --deploy --dev
pipenv run python -m spacy download en_core_web_lg
- template: ./build-python.yml
parameters:
Expand Down
22 changes: 0 additions & 22 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,28 +77,6 @@ Follow these steps when starting to work on a Presidio service with Pipenv:
pip freeze
```
#### Dependency lock using Pipenv
In some cases we would like to lock the dependencies used by each package. To lock dependencies, first update the Pipfile file, then run the following command:
```sh
pipenv lock
```
Lock considerations:
1. Consider running `pipenv lock` on a Python 3.8 environment. Using Python 3.7 makes sure the lock would apply to all Python versions supported by Presidio (3.7 and on).
2. Consider locking on mac/linux and not Windows to support all platforms. In windows you can use [WSL](https://docs.microsoft.com/en-us/windows/wsl/install-win10).
For example, use conda to create a Python 3.8 environment:
```sh
conda create -n presidio_38 python=3.8
conda activate presidio_38
# in the package's directory:
pipenv lock
```
### Development guidelines
- A Github issue suggesting the change should be opened prior to a PR.
Expand Down
2 changes: 1 addition & 1 deletion presidio-analyzer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ WORKDIR /usr/bin/${NAME}

COPY ./Pipfile* /usr/bin/${NAME}/
RUN pip install pipenv \
&& pipenv sync
&& pipenv install --deploy
# install nlp models specified in conf/default.yaml
COPY ./install_nlp_models.py /usr/bin/${NAME}/
COPY ${NLP_CONF_FILE} /usr/bin/${NAME}/${NLP_CONF_FILE}
Expand Down
2 changes: 1 addition & 1 deletion presidio-analyzer/Dockerfile.transformers
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ WORKDIR /usr/bin/${NAME}

COPY ./Pipfile* /usr/bin/${NAME}/
RUN pip install pipenv \
&& pipenv sync
&& pipenv install --deploy
RUN pipenv install torch transformers huggingface_hub --skip-lock

# install nlp models specified in conf/default.yaml
Expand Down
4 changes: 2 additions & 2 deletions presidio-analyzer/Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ verify_ssl = true
name = "pypi"

[packages]
spacy = ">=3.4.4"
spacy = ">=3.4.4,<4.0.0"
regex = "*"
tldextract = "*"
flask = ">=1.1"
pyyaml = "*"
phonenumbers = ">=8.12"
phonenumbers = ">=8.12,<9.0.0"
typing-extensions = "*"

[dev-packages]
Expand Down
1,044 changes: 0 additions & 1,044 deletions presidio-analyzer/Pipfile.lock

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ def _get_nlp_recognizer(
) -> Union[Type[SpacyRecognizer], Type[StanzaRecognizer]]:
"""Return the recognizer leveraging the selected NLP Engine."""

if not nlp_engine or type(nlp_engine) == SpacyNlpEngine:
if not nlp_engine or type(nlp_engine) is SpacyNlpEngine:
return SpacyRecognizer
if isinstance(nlp_engine, StanzaNlpEngine):
return StanzaRecognizer
Expand Down
2 changes: 1 addition & 1 deletion presidio-anonymizer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ WORKDIR /usr/bin/${NAME}

COPY ./Pipfile* /usr/bin/${NAME}/
RUN pip install pipenv \
&& pipenv sync
&& pipenv install --deploy

COPY . /usr/bin/${NAME}/

Expand Down
2 changes: 1 addition & 1 deletion presidio-anonymizer/Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ name = "pypi"

[packages]
flask = ">=1.1"
pycryptodome = ">=3.10"
pycryptodome = ">=3.10,<4.0.0"

[dev-packages]
pytest = "*"
Expand Down
Loading

0 comments on commit 18090c6

Please sign in to comment.