Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
"""The application-level conftest."""
|
|
|
|
import asyncio
|
|
|
|
import contextlib
|
|
|
|
from collections.abc import AsyncGenerator, Generator
|
|
|
|
from typing import Literal
|
|
|
|
|
|
|
|
import pytest
|
|
|
|
import stamina
|
|
|
|
from dirty_equals import IsList
|
2023-11-18 19:27:38 +00:00
|
|
|
from sqlalchemy.ext.asyncio import (
|
|
|
|
AsyncConnection,
|
|
|
|
AsyncEngine,
|
|
|
|
AsyncSession,
|
|
|
|
async_sessionmaker,
|
|
|
|
create_async_engine,
|
|
|
|
)
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
|
|
|
|
from app.database import Dependency, Repo
|
|
|
|
from app.factories import DependencyCreateDataFactory
|
|
|
|
from app.source_graph.factories import SourceGraphRepoDataFactory
|
|
|
|
from app.source_graph.models import SourceGraphRepoData
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(autouse=True, scope="session")
|
|
|
|
def anyio_backend() -> Literal["asyncio"]:
|
|
|
|
"""Use asyncio as the async backend."""
|
|
|
|
return "asyncio"
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(autouse=True, scope="session")
|
|
|
|
def _deactivate_retries() -> None:
|
|
|
|
"""Deactivate stamina retries."""
|
|
|
|
stamina.set_active(False)
|
|
|
|
|
|
|
|
|
2023-11-18 19:27:38 +00:00
|
|
|
@pytest.fixture(scope="session")
|
|
|
|
def db_path() -> str:
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
"""Use the in-memory database for tests."""
|
2023-11-18 19:27:38 +00:00
|
|
|
return "" # ":memory:"
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
|
|
def db_connection_string(
|
|
|
|
db_path: str,
|
|
|
|
) -> str:
|
|
|
|
"""Provide the connection string for the in-memory database."""
|
|
|
|
return f"sqlite+aiosqlite:///{db_path}"
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(scope="session", params=[{"echo": False}], ids=["echo=False"])
|
|
|
|
async def db_engine(
|
|
|
|
db_connection_string: str,
|
|
|
|
request: pytest.FixtureRequest,
|
|
|
|
) -> AsyncGenerator[AsyncEngine, None, None]:
|
|
|
|
"""Create the database engine."""
|
|
|
|
# echo=True enables logging of all SQL statements
|
|
|
|
# https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.create_engine.params.echo
|
|
|
|
engine = create_async_engine(
|
|
|
|
db_connection_string,
|
|
|
|
**request.param, # type: ignore
|
|
|
|
)
|
|
|
|
try:
|
|
|
|
yield engine
|
|
|
|
finally:
|
|
|
|
# for AsyncEngine created in function scope, close and
|
|
|
|
# clean-up pooled connections
|
|
|
|
await engine.dispose()
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
|
|
def event_loop(
|
|
|
|
request: pytest.FixtureRequest,
|
|
|
|
) -> Generator[asyncio.AbstractEventLoop, None, None]:
|
|
|
|
"""
|
|
|
|
Create an instance of the default event loop for a session.
|
|
|
|
|
|
|
|
An event loop is destroyed at the end of the test session.
|
|
|
|
https://docs.pytest.org/en/6.2.x/fixture.html#fixture-scopes
|
|
|
|
"""
|
|
|
|
with contextlib.closing(loop := asyncio.get_event_loop_policy().get_event_loop()):
|
|
|
|
yield loop
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
2023-11-18 19:27:38 +00:00
|
|
|
async def _database_objects(
|
|
|
|
db_engine: AsyncEngine,
|
|
|
|
) -> AsyncGenerator[None, None]:
|
|
|
|
"""Create the database objects (tables, etc.)."""
|
|
|
|
from app.database import Base
|
|
|
|
|
|
|
|
# Enters a transaction
|
|
|
|
# https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.AsyncConnection.begin
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
try:
|
2023-11-18 19:27:38 +00:00
|
|
|
async with db_engine.begin() as conn:
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
await conn.run_sync(Base.metadata.drop_all)
|
|
|
|
await conn.run_sync(Base.metadata.create_all)
|
2023-11-18 19:27:38 +00:00
|
|
|
yield
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
finally:
|
2023-11-18 19:27:38 +00:00
|
|
|
# Clean up after the testing session is over
|
|
|
|
async with db_engine.begin() as conn:
|
|
|
|
await conn.run_sync(Base.metadata.drop_all)
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
|
|
async def db_connection(
|
|
|
|
db_engine: AsyncEngine,
|
|
|
|
) -> AsyncGenerator[AsyncConnection, None]:
|
|
|
|
"""Create a database connection."""
|
|
|
|
# Return connection with no transaction
|
|
|
|
# https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.AsyncEngine.connect
|
|
|
|
async with db_engine.connect() as conn:
|
|
|
|
yield conn
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture()
|
2023-11-18 19:27:38 +00:00
|
|
|
async def db_session(
|
|
|
|
db_engine: AsyncEngine,
|
|
|
|
_database_objects: None,
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
) -> AsyncGenerator[AsyncSession, None]:
|
2023-11-18 19:27:38 +00:00
|
|
|
"""Create a database session."""
|
|
|
|
# The `async_sessionmaker` function is used to create a Session factory
|
|
|
|
# https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker
|
|
|
|
async_session_factory = async_sessionmaker(
|
|
|
|
db_engine, expire_on_commit=False, autoflush=False, autocommit=False
|
|
|
|
)
|
|
|
|
async with async_session_factory() as session:
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
yield session
|
|
|
|
|
|
|
|
|
2023-11-18 19:27:38 +00:00
|
|
|
@pytest.fixture()
|
|
|
|
async def db_uow(
|
|
|
|
db_session: AsyncSession,
|
|
|
|
) -> AsyncGenerator[AsyncSession, None]:
|
|
|
|
"""Provide a transactional scope around a series of operations."""
|
2023-11-19 22:17:57 +00:00
|
|
|
from app.uow import async_session_uow
|
|
|
|
|
2023-11-18 19:27:38 +00:00
|
|
|
# This context manager will start a transaction, and roll it back at the end
|
|
|
|
# https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.AsyncSessionTransaction
|
2023-11-19 22:17:57 +00:00
|
|
|
async with async_session_uow(db_session) as session:
|
|
|
|
yield session
|
2023-11-18 19:27:38 +00:00
|
|
|
|
|
|
|
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
@pytest.fixture()
|
|
|
|
async def some_repos(
|
2023-11-18 19:27:38 +00:00
|
|
|
db_session: AsyncSession,
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
source_graph_repo_data_factory: SourceGraphRepoDataFactory,
|
|
|
|
dependency_create_data_factory: DependencyCreateDataFactory,
|
|
|
|
) -> list[Repo]:
|
|
|
|
"""Create some repos."""
|
|
|
|
source_graph_repos_data: list[
|
|
|
|
SourceGraphRepoData
|
|
|
|
] = source_graph_repo_data_factory.batch(10)
|
|
|
|
assert source_graph_repos_data == IsList(length=10)
|
|
|
|
repos = [
|
|
|
|
Repo(
|
|
|
|
url=str(source_graph_repo_data.repo_url),
|
|
|
|
description=source_graph_repo_data.description,
|
|
|
|
stars=source_graph_repo_data.stars,
|
|
|
|
source_graph_repo_id=source_graph_repo_data.repo_id,
|
|
|
|
dependencies=[
|
|
|
|
Dependency(**dependency_create_data.model_dump())
|
|
|
|
for dependency_create_data in dependency_create_data_factory.batch(5)
|
|
|
|
],
|
|
|
|
)
|
|
|
|
for source_graph_repo_data in source_graph_repos_data
|
|
|
|
]
|
2023-11-18 19:27:38 +00:00
|
|
|
db_session.add_all(repos)
|
|
|
|
await db_session.flush()
|
Web App (#25)
* Set up the web project dependencies
- Add linters, pre-commit, and GitHub Actions
- Add a Makefile
- Add a pyproject.toml
* Fix pyupgrade job
- Remove continue_on_error everywhere
* Remove old code
* Rename a GithubActions job
* Change README
* Adjust pre-commit and GitHub actions
* Add tables and set up alembic
* Set up tests
* Extend tests
* Add coverage config
* Adjust the GithubActions workflow
* Fix GithubActions workflow
* Try fixing pyproject-fmt config
* Fix formatting of pyproject.toml
* Fix formatting of pyproject.toml
* Add coverage report
* Test listing the repositories
* Add a working prototype of SourceGraph client
* Add parsing of the SourceGraph SSE data
* Fix tests
* Ged rid of packages replaced by ruff
* Fix waits in the SourceGraph client
* Refactor the models and add a mapper
- A new mapper allows to create database repositories from the SourceGraph data
* Add mypy
* Try fixing mypy action
* Remove redundant configs
* Exclude tests from type checking
* Fix mypy pre-commit and GitHub action
* Ignore factories
* Make upserting possible for source graph data
* Add logic for parsing the dependencies and populating the database
* Add a database and a cron GitHub Action job
* Try manually trigger a workflow
* Bring back the old config
* Add ReadTimeout for errors to retry for in SourceGraph client
* Add typer
* Adjust the docstrings
* Update the database
* Refactor and optimize scraping and dependencies parsing
* Make scraping run on push for now
* Add a unique constraint for the repo url and source graph repo id
* Change the index columns in on_conflict statement for repo creation
* Optimize dependencies parsing
- Do not parse dependencies for a repo when revision did not change
* Scraped repositories from Source Graph
* Refactor scraping
* Set up frontend
* Scraped repositories from Source Graph
* Add TODOs
* Skip scraping when testing
* Fix a test with updating the repos
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Add some more TODO comments
* Add chadcn/ui
* Scraped repositories from Source Graph
* Create index.json
* Scraped repositories from Source Graph
* Add a draft of data table and display all the repos
* Scraped repositories from Source Graph
* Implement stars badges and description with overflow
* Format the links to Github repos
* Fix link clicking
* Scraped repositories from Source Graph
* Add simple pagination and stars column sorting
* Scraped repositories from Source Graph
* Implement basic searching
* Scraped repositories from Source Graph
* Implement a multiselect for dependencies
* Scraped repositories from Source Graph
* Implement actual filtering by dependencies
* Scraped repositories from Source Graph
* Add a workflow to deploy nextjs on github pages
* Try fixing the deployment job
* Enable static exports for app router
* Fix uploading arifacts for nextjs job
* Set base path to properly load JS and CSS
* Fix the base path
* Scraped repositories from Source Graph
* Add header
* Remove language version
* Scraped repositories from Source Graph
* Add some more TODOs
* Scraped repositories from Source Graph
* Adjust the pre-commit config
* Fix pipelines
* Scraped repositories from Source Graph
* Add a footer
* Create the indexes
* Scraped repositories from Source Graph
* Add more TODOs
* Introduce minor footer adjustments
* Adjust the scraping actions
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Implement query params state
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Do not commit query state on unmount
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Hopefully fix query states and multiselect input
* Scraped repositories from Source Graph, parsed the dependencies, and generated the indexes
* Extend the Makefile
* Resolve most of TODOs
* Resolve the conflicts with anyio version, bring back httpx
* Adjust the Makefile and README.md
* Fix a typo in README.md
* Adjust readme
* Fix the Makefile
* Fix some stuff
* Make some adjustments
* Possibly fix failing scraping jobs
* Load the repo project URL from env
---------
Co-authored-by: vladfedoriuk <vladfedoriuk@users.noreply.github.com>
Co-authored-by: Vladyslav Fedoriuk <vladyslav.fedoriuk@deployed.pl>
2023-10-28 19:39:02 +00:00
|
|
|
return repos
|