diff --git a/docs/index.rst b/docs/index.rst index 91c514f..33441a3 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -39,7 +39,7 @@ Document Structure .. toctree:: :maxdepth: 3 - :caption: CONCEPTS: + :caption: COMPONENTS: Market Learnware & Reuser diff --git a/docs/workflows/client.rst b/docs/workflows/client.rst index 5f0ea5b..b76fb8e 100644 --- a/docs/workflows/client.rst +++ b/docs/workflows/client.rst @@ -6,13 +6,13 @@ Learnware Client Introduction ==================== -``Learnware Client`` is a python api that provides a convenient interface for interacting with the official market. You can easily use the client to upload, download and search learnwares. +``Learnware Client`` is a ``Python API`` that provides a convenient interface for interacting with the ``BeimingWu`` system. You can easily use the client to upload, download, delete, update, and search learnwares. Prepare access token ==================== -Before using the ``Learnware Client``, you'll need to obtain a token from the `official website `_. Just login to the website and click "client token" tab in the user center. +Before using the ``Learnware Client``, you'll need to obtain a token from the `official website `_. Just login to the website and click ``Client Token`` tab in the ``Personal Information``. How to Use Client @@ -25,142 +25,209 @@ Initialize a Learware Client .. code-block:: python - import learnware - from learnware.client import LearnwareClient + from learnware.client import LearnwareClient, SemanticSpecificationKey + # Login to Beiming system client = LearnwareClient() - - # login to official market client.login(email="your email", token="your token") +Where email is the registered mailbox of the system and token is the token obtained in the previous section. Upload Leanware ------------------------------- -Before uploading a learnware, you'll need to prepare the semantic specification of your learnware. You can create a semantic specification by a helper function ``create_semantic_specification``. +Before uploading a learnware, you'll need to prepare the semantic specification of your learnware. Let's take the classification task for tabular data as an example. You can create a semantic specification by a helper function ``create_semantic_specification``. .. code-block:: python + # Prepare input description when data_type="Table" input_description = { - "Dimension": 16, - { - "Description": { - "0": "gender", - "1": "age", - "2": "f2", - "5": "f5" - } - } + "Dimension": 5, + "Description": { + "0": "age", + "1": "weight", + "2": "body length", + "3": "animal type", + "4": "claw length" + }, } + + # Prepare output description when task_type in ["Classification", "Regression"] output_description = { "Dimension": 3, "Description": { - "0": "the probability of being a cat", - "1": "the probability of being a dog", - "2": "the probability of being a bird" - } + "0": "cat", + "1": "dog", + "2": "bird", + }, } + + # Create semantic specification semantic_spec = client.create_semantic_specification( - name="mylearnware1", - description="this is my learnware", - data_type="Table", - task_type="Classification", - library_type="Scikit-learn", - senarioes=["Business", "Financial"], - input_description, output_description) - # data_type, task_type, library_type, senarioes are enums, you can find possible values in `learnware.C` + name="learnware_example", + description="Just a example for uploading a learnware", + data_type="Table", + task_type="Classification", + library_type="Scikit-learn", + scenarios=["Business", "Financial"], + license=["Apache-2.0"], + input_description=input_description, + output_description=output_description, + ) +Ensure that the input parameters for the semantic specification fall within the specified ranges provided by ``client.list_semantic_specification_values(key)``: + +* "data_type" must be within the range of ``key=SemanticSpecificationKey.DATA_TYPE``. +* "task_type" must be within the range of ``key=SemanticSpecificationKey.TASK_TYPE``. +* "library_type" must be within the range of ``key=SemanticSpecificationKey.LIBRARY_TYPE``. +* "scenarios" must be a subset of ``key=SemanticSpecificationKey.SENARIOS``. +* "license" must be a subset of ``key=SemanticSpecificationKey.LICENSE``. +* When "data_type" is set to "Table", it is necessary to provide "input_description". +* When "task_type" is either "Classification" or "Regression", it is necessary to provide "output_description". + +Finally, the semantic specification and the zip package path of the learnware were filled in to upload the learnware. + +Remember to verify the learnware before uploading it, as shown in the following code example: -After defining the semantic specification, -you can upload your learnware using ``upload_learnware`` function: - .. code-block:: python - - learnware_id = client.upload_learnware( - semantic_spec=semantic_spec, - zip_path="path to your learnware zipfile") -Here, ``zip_path`` is the local path of your learnware zipfile. + # Prepare your learnware zip file + zip_path = "your learnware zip" + # Check your learnware before upload + client.check_learnware( + learnware_zip_path=zip_path, semantic_specification=semantic_spec + ) -Semantic Specification Search + # Upload your learnware + learnware_id = client.upload_learnware( + learnware_zip_path=zip_path, semantic_specification=semantic_spec + ) + +After uploading the learnware successfully, you can see it in ``My Learnware``, the background will check it. Click on the learnware, which can be viewed in the ``Verify Status``. After the check passes, the Unverified tag of the learnware will disappear, and the uploaded learnware will appear in the system. + +Update Learnware ------------------------------- -You can search learnwares in official market using semantic specification. All the learnwares that match the semantic specification will be returned by the api. For example, the code below searches learnwares with `Table` data type: +The ``update_learnware`` method is used to update the metadata and content of an existing learnware on the server. You can upload a new semantic specification, or directly upload a new learnware. .. code-block:: python + # Replace with the actual learnware ID + learnware_id = "123456789" + + # Create new semantic specification semantic_spec = client.create_semantic_specification( - name="", - description="", - data_type="Table", - task_type="", - library_type="", - senarioes=[], - input_description={}, output_description={}) - - specification = learnware.specification.Specification() - specification.update_semantic_spec(specification) - learnware_list = client.search_learnware(specification) - + name="new learnware name", + description="new description", + data_type="Table", + task_type="Classification", + library_type="Scikit-learn", + scenarios=["Computer", "Internet"], + license=["CC-BY-4.0"], + input_description=new_input_description, + output_description=new_output_description, + ) + + # Update metadata without changing the content + client.update_learnware(learnware_id, semantic_spec) + + # Update metadata and content with a new ZIP file + updated_zip_path = "/path/to/updated_learnware.zip" + client.update_learnware(learnware_id, semantic_spec, learnware_zip_path=updated_zip_path) + +Delete Learnware +------------------------------- -Statistical Specification Search ---------------------------------- +The ``delete_learnware`` method is used to delete a learnware from the server. + +.. code-block:: python + + # Replace with the actual learnware ID to delete + learnware_id = "123456789" + + # Delete the specified learnware + client.delete_learnware(learnware_id) -You can search learnware by providing a statistical specification. The statistical specification is a json file that contains the statistical information of your training data. For example, the code below searches learnwares with `RKMETableSpecification`: + +Semantic Specification Search +------------------------------- + +You can search the learnware in the system through the semantic specification, and all the learnware conforming to the semantic specification will be returned through the API. For example, the following code will give you all the learnware in the system whose task type is classified: .. code-block:: python - import learnware.specification as specification + from learnware.market import BaseUserInfo - user_spec = specification.RKMETableSpecification() - user_spec.load(os.path.join(unzip_path, "rkme.json")) + user_semantic = client.create_semantic_specification( + task_type="Classification" + ) + user_info = BaseUserInfo(semantic_spec=user_semantic) + learnware_list = client.search_learnware(user_info, page_size=None) - specification = learnware.specification.Specification() - specification.update_stat_spec(user_spec) - learnware_list = client.search_learnware(specification) +Statistical Specification Search +--------------------------------- - # you can view the scores of the searched learnwares - for learnware in learnware_list: - print(f'learnware_id: {learnware["learnware_id"]}, score: {learnware["matching"]}') +You can also search the learnware in the system through the statistical specification, and all the learnware with similar distribution will be returned through the API. Using the ``generate_stat_spec`` function mentioned above, you can easily get the ``stat_spec`` for your current task, and then get the learnware that meets the statistical specification for the same type of data in the system by using the following code: + +.. code-block:: python + + user_info = BaseUserInfo(stat_info={stat_spec.type: stat_spec}) + learnware_list = client.search_learnware(user_info, page_size=None) Combine Semantic and Statistical Search ---------------------------------------- -You can provide both semantic and statistical specification to search learnwares. The engine will first filter learnwares by semantic specification and then search by statistical specification. For example, the code below searches learnwares with `Table` data type and `RKMETableSpecification`: +By combining statistical and semantic specifications, you can perform more detailed searches, such as the following code that searches tabular data for pieces of learnware that satisfy your semantic specifications: .. code-block:: python - semantic_spec = client.create_semantic_specification( - name="", - description="", - data_type="Table", - task_type="", - library_type="", - senarioes=[], - input_description={}, output_description={}) + user_semantic = client.create_semantic_specification( + task_type="Classification", + scenarios=["Business"], + ) + rkme_table = generate_stat_spec(type="table", X=train_x) + user_info = BaseUserInfo( + semantic_spec=user_semantic, stat_info={rkme_table.type: rkme_table} + ) + learnware_list = client.search_learnware(user_info, page_size=None) + +Heterogeneous Table Search +---------------------------------------- +When you provide a statistical specification for tabular data, the task type is "Classification" or "Regression", and your semantic specification includes descriptions for each dimension, the system will automatically enable heterogeneous table search. It won't only search in the tabular learnwares with same dimensions. The following code will perform heterogeneous table search through the API: - stat_spec = specification.RKMETableSpecification() - stat_spec.load(os.path.join(unzip_path, "rkme.json")) - specification = learnware.specification.Specification() - specification.update_semantic_spec(semantic_spec) - specification.update_stat_spec(stat_spec) +.. code-block:: python - learnware_list = client.search_learnware(specification) + input_description = { + "Dimension": 2, + "Description": { + "0": "leaf width", + "1": "leaf length", + }, + } + user_semantic = client.create_semantic_specification( + task_type="Classification", + scenarios=["Business"], + input_description=input_description, + ) + rkme_table = generate_stat_spec(type="table", X=train_x) + user_info = BaseUserInfo( + semantic_spec=user_semantic, stat_info={rkme_table.type: rkme_table} + ) + learnware_list = client.search_learnware(user_info) Download and Use Learnware ------------------------------- -When you get a learnware id, you can download and initiate the learnware with the following code: +When the search is complete, you can download the learnware and configure the environment through the following code: .. code-block:: python - client.download_learnware(learnware_id, zip_path) - client.install_environment(zip_path) - learnware = client.load_learnware(zip_path) - # you can use the learnware to make prediction now - - - + for temp_learnware in learnware_list: + learnware_id = temp_learnware["learnware_id"] + # you can use the learnware to make prediction now + learnware = client.load_learnware( + learnware_id=learnware_id, runnable_option="conda" + ) \ No newline at end of file diff --git a/examples/dataset_image_workflow/main.py b/examples/dataset_image_workflow/main.py index c91981c..0512c31 100644 --- a/examples/dataset_image_workflow/main.py +++ b/examples/dataset_image_workflow/main.py @@ -50,6 +50,7 @@ semantic_specs = [ "Description": {"Values": "", "Type": "String"}, "Name": {"Values": "learnware_1", "Type": "String"}, "Output": {"Dimension": 10}, + "License": {"Values": ["MIT"], "Type": "Class"}, } ] @@ -60,6 +61,7 @@ user_semantic = { "Scenario": {"Values": ["Business"], "Type": "Tag"}, "Description": {"Values": "", "Type": "String"}, "Name": {"Values": "", "Type": "String"}, + "License": {"Values": ["MIT"], "Type": "Class"}, } @@ -175,7 +177,10 @@ def test_search(gamma=0.1, load_market=True): pred_y = single_item.learnware.predict(user_data) acc = eval_prediction(pred_y, user_label) acc_list.append(acc) - logger.info("Search rank: %d, score: %.3f, learnware_id: %s, acc: %.3f" % (idx, single_item.score, single_item.learnware.id, acc)) + logger.info( + "Search rank: %d, score: %.3f, learnware_id: %s, acc: %.3f" + % (idx, single_item.score, single_item.learnware.id, acc) + ) # test reuse (job selector) # reuse_baseline = JobSelectorReuser(learnware_list=mixture_learnware_list, herding_num=100) diff --git a/examples/dataset_m5_workflow/main.py b/examples/dataset_m5_workflow/main.py index 60ee439..c6dea40 100644 --- a/examples/dataset_m5_workflow/main.py +++ b/examples/dataset_m5_workflow/main.py @@ -36,6 +36,7 @@ semantic_specs = [ "Name": {"Values": "learnware_1", "Type": "String"}, "Input": input_description, "Output": output_description, + "License": {"Values": ["MIT"], "Type": "Class"}, } ] @@ -48,6 +49,7 @@ user_semantic = { "Name": {"Values": "", "Type": "String"}, "Input": input_description, "Output": output_description, + "License": {"Values": ["MIT"], "Type": "Class"}, } @@ -158,7 +160,7 @@ class M5DatasetWorkflow: search_result = easy_market.search_learnware(user_info) single_result = search_result.get_single_results() multiple_result = search_result.get_multiple_results() - + print(f"search result of user{idx}:") print( f"single model num: {len(single_result)}, max_score: {single_result[0].score}, min_score: {single_result[-1].score}" diff --git a/examples/dataset_m5_workflow/upload.py b/examples/dataset_m5_workflow/upload.py index 0c9e209..af0a69d 100644 --- a/examples/dataset_m5_workflow/upload.py +++ b/examples/dataset_m5_workflow/upload.py @@ -66,6 +66,7 @@ def main(): "Scenario": {"Values": ["Business"], "Type": "Tag"}, "Description": {"Values": "A sales-forecasting model from Walmart store", "Type": "String"}, "Name": {"Values": name, "Type": "String"}, + "License": {"Values": ["MIT"], "Type": "Class"}, } res = session.post( submit_url, diff --git a/examples/dataset_pfs_workflow/main.py b/examples/dataset_pfs_workflow/main.py index 74c4da5..2c33f04 100644 --- a/examples/dataset_pfs_workflow/main.py +++ b/examples/dataset_pfs_workflow/main.py @@ -35,6 +35,7 @@ semantic_specs = [ "Name": {"Values": "learnware_1", "Type": "String"}, "Input": input_description, "Output": output_description, + "License": {"Values": ["MIT"], "Type": "Class"}, } ] @@ -47,6 +48,7 @@ user_semantic = { "Name": {"Values": "", "Type": "String"}, "Input": input_description, "Output": output_description, + "License": {"Values": ["MIT"], "Type": "Class"}, } @@ -155,7 +157,7 @@ class PFSDatasetWorkflow: search_result = easy_market.search_learnware(user_info) single_result = search_result.get_single_results() multiple_result = search_result.get_multiple_results() - + print(f"search result of user{idx}:") print( f"single model num: {len(single_result)}, max_score: {single_result[0].score}, min_score: {single_result[-1].score}" diff --git a/examples/dataset_pfs_workflow/upload.py b/examples/dataset_pfs_workflow/upload.py index 9719230..c9da3db 100644 --- a/examples/dataset_pfs_workflow/upload.py +++ b/examples/dataset_pfs_workflow/upload.py @@ -69,6 +69,7 @@ def main(): "Type": "String", }, "Name": {"Values": name, "Type": "String"}, + "License": {"Values": ["MIT"], "Type": "Class"}, } res = session.post( submit_url, diff --git a/examples/dataset_text_workflow/main.py b/examples/dataset_text_workflow/main.py index c5715e7..dcb3898 100644 --- a/examples/dataset_text_workflow/main.py +++ b/examples/dataset_text_workflow/main.py @@ -48,6 +48,7 @@ semantic_specs = [ "Description": {"Values": "", "Type": "String"}, "Name": {"Values": "learnware_1", "Type": "String"}, "Output": output_description, + "License": {"Values": ["MIT"], "Type": "Class"}, } ] @@ -59,6 +60,7 @@ user_semantic = { "Description": {"Values": "", "Type": "String"}, "Name": {"Values": "", "Type": "String"}, "Output": output_description, + "License": {"Values": ["MIT"], "Type": "Class"}, } @@ -73,7 +75,6 @@ class TextDatasetWorkflow: generate_uploader(X_train, y_train, n_uploaders=n_uploaders, data_save_root=uploader_save_root) generate_user(X_test, y_test, n_users=n_users, data_save_root=user_save_root) - def _prepare_model(self): dataloader = TextDataLoader(data_save_root, train=True) for i in range(n_uploaders): @@ -92,9 +93,8 @@ class TextDatasetWorkflow: logger.info("Model saved to '%s' and '%s'" % (modelv_save_path, modell_save_path)) - - def _prepare_learnware(self, - data_path, modelv_path, modell_path, init_file_path, yaml_path, env_file_path, save_root, zip_name + def _prepare_learnware( + self, data_path, modelv_path, modell_path, init_file_path, yaml_path, env_file_path, save_root, zip_name ): os.makedirs(save_root, exist_ok=True) tmp_spec_path = os.path.join(save_root, "rkme.json") @@ -139,7 +139,6 @@ class TextDatasetWorkflow: logger.info("New Learnware Saved to %s" % (zip_file_name)) return zip_file_name - def prepare_market(self, regenerate_flag=False): if regenerate_flag: self._init_text_dataset() @@ -175,7 +174,6 @@ class TextDatasetWorkflow: logger.info("Total Item: %d" % (len(text_market))) - def test(self, regenerate_flag=False): self.prepare_market(regenerate_flag) text_market = instantiate_learnware_market(market_id="ae") @@ -199,11 +197,11 @@ class TextDatasetWorkflow: user_stat_spec.generate_stat_spec_from_data(X=user_data) user_info = BaseUserInfo(semantic_spec=user_semantic, stat_info={"RKMETextSpecification": user_stat_spec}) logger.info("Searching Market for user: %d" % (i)) - + search_result = text_market.search_learnware(user_info) single_result = search_result.get_single_results() multiple_result = search_result.get_multiple_results() - + print(f"search result of user{i}:") print( f"single model num: {len(single_result)}, max_score: {single_result[0].score}, min_score: {single_result[-1].score}" @@ -220,7 +218,7 @@ class TextDatasetWorkflow: print( f"Top1-score: {single_result[0].score}, learnware_id: {single_result[0].learnware.id}, acc: {acc_list[0]}" ) - + if len(multiple_result) > 0: mixture_id = " ".join([learnware.id for learnware in multiple_result[0].learnwares]) print(f"mixture_score: {multiple_result[0].score}, mixture_learnware: {mixture_id}") diff --git a/learnware/__init__.py b/learnware/__init__.py index c3f122e..eed5dde 100644 --- a/learnware/__init__.py +++ b/learnware/__init__.py @@ -1,4 +1,4 @@ -__version__ = "0.2.0.4" +__version__ = "0.2.0.5" import os import json diff --git a/learnware/client/container.py b/learnware/client/container.py index 5573799..590af59 100644 --- a/learnware/client/container.py +++ b/learnware/client/container.py @@ -43,16 +43,19 @@ class ModelContainer(BaseModel): """We must set `input_shape` and `output_shape`""" if self.build: self.cleanup_flag = True - self._init_env() atexit.register(self.remove_env) - + self._init_env() self._setup_env_and_metadata() def remove_env(self): if self.cleanup_flag is True: - self.cleanup_flag = False try: + self.cleanup_flag = False self._remove_env() + except KeyboardInterrupt: + self.cleanup_flag = True + logger.warning("The KeyboardInterrupt is ignored when removing the container env!") + self.remove_env() except Exception as err: self.cleanup_flag = True raise err @@ -106,7 +109,7 @@ class ModelCondaContainer(ModelContainer): "-n", f"{self.conda_env}", "--no-capture-output", - "python3", + "python", f"{self.model_script}", "--model-path", f"{model_path}", @@ -149,7 +152,7 @@ class ModelCondaContainer(ModelContainer): "-n", f"{self.conda_env}", "--no-capture-output", - "python3", + "python", f"{self.model_script}", "--model-path", f"{model_path}", @@ -333,7 +336,7 @@ class ModelDockerContainer(ModelContainer): "-n", f"{conda_env}", "--no-capture-output", - "python3", + "python", "-m", "pip", "install", @@ -362,7 +365,7 @@ class ModelDockerContainer(ModelContainer): "-n", f"{conda_env}", "--no-capture-output", - "python3", + "python", "-m", "pip", "install", @@ -404,7 +407,7 @@ class ModelDockerContainer(ModelContainer): "-n", f"{self.conda_env}", "--no-capture-output", - "python3", + "python", f"{self.docker_model_script_path}", "--model-path", f"{model_path}", @@ -458,7 +461,7 @@ class ModelDockerContainer(ModelContainer): "-n", f"{self.conda_env}", "--no-capture-output", - "python3", + "python", f"{self.docker_model_script_path}", "--model-path", f"{model_path}", @@ -553,14 +556,13 @@ class LearnwaresContainer: if sum(self.results) < len(self.learnware_list): logger.warning( - f"{len(self.learnware_list) - sum(results)} of {len(self.learnware_list)} learnwares init failed! This learnware will be ignored" + f"{len(self.learnware_list) - sum(results)} of {len(self.learnware_list)} learnwares init failed! These learnwares will be ignored" ) return self def __exit__(self, exc_type, exc_val, exc_tb): if not self.cleanup: - logger.warning(f"Notice, the learnware container env is not cleaned up!") self.learnware_containers = None self.results = None return diff --git a/learnware/client/package_utils.py b/learnware/client/package_utils.py index 5768a4e..f3266ae 100644 --- a/learnware/client/package_utils.py +++ b/learnware/client/package_utils.py @@ -6,6 +6,7 @@ import tempfile import subprocess from typing import List, Tuple from . import utils +from concurrent.futures import ThreadPoolExecutor from ..logger import get_module_logger @@ -13,38 +14,38 @@ from ..logger import get_module_logger logger = get_module_logger("package_utils") -def try_to_run(args, timeout=5, retry=5): - sucess = False +def try_to_run(args, timeout=10, retry=3): for i in range(retry): try: - utils.system_execute(args=args, timeout=timeout, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) - sucess = True - break + result = utils.system_execute(args=args, timeout=timeout, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL) + return result.stdout.decode() except subprocess.TimeoutExpired as e: pass - - if not sucess: - raise subprocess.TimeoutExpired(args, timeout) + + raise subprocess.TimeoutExpired(args, timeout) def parse_pip_requirement(line: str): - """Parse pip requirement line to package name""" + """Parse pip requirement line to package name and version""" line = line.strip() - - if len(line) == 0: - return None - - if line[0] in ("#", "-"): + if len(line) == 0 or line[0] in ("#", "-"): return None - package_str = line - for split_ch in ("=", ">", "<", "!", "~", " "): - split_ch_index = package_str.find(split_ch) + package_name, package_version = line, line + for split_ch in ("=", ">", "<", "!", "~", " ", "="): + split_ch_index = package_name.find(split_ch) if split_ch_index != -1: - package_str = package_str[:split_ch_index] + package_name = package_name[:split_ch_index] + + split_ch_index = package_version.find(split_ch) + if split_ch_index != -1: + package_version = package_version[split_ch_index + 1:] + + if package_version == package_name: + package_version = "" - return package_str + return package_name, package_version def read_pip_packages_from_requirements(requirements_file: str) -> Tuple[List[str], List[str]]: @@ -54,7 +55,7 @@ def read_pip_packages_from_requirements(requirements_file: str) -> Tuple[List[st lines = [] with open(requirements_file, "r") as fin: for line in fin: - package_str = parse_pip_requirement(line) + package_str, package_version = parse_pip_requirement(line) packages.append(package_str) lines.append(line) @@ -70,22 +71,34 @@ def filter_nonexist_pip_packages(packages: list) -> Tuple[List[str], List[str]]: exist_packages: list of exist packages nonexist_packages: list of non-exist packages """ - - exist_packages = [] - nonexist_packages = [] - for package in packages: - if package is None: - continue + def _filter_nonexist_pip_package_worker(package): + # Return filtered package try: - package_name = parse_pip_requirement(package) + package_name, package_version = parse_pip_requirement(package) if package_name is not None and package_name != "learnware": - try_to_run(args=["pip", "index", "versions", package_name], timeout=5) - exist_packages.append(package) - continue + result = try_to_run(args=["pip", "index", "versions", package_name], timeout=10) + if len(package_version) and package_version not in result: + return package_name + else: + return package except Exception as e: logger.error(e) - nonexist_packages.append(package) - + + return None + + exist_packages = [] + nonexist_packages = [] + packages = [package for package in packages if package is not None] + + with ThreadPoolExecutor(max_workers=max(os.cpu_count() // 5, 1)) as executor: + results = executor.map(_filter_nonexist_pip_package_worker, packages) + + for result, package in zip(list(results), packages): + if result is not None: + exist_packages.append(result) + else: + nonexist_packages.append(package) + return exist_packages, nonexist_packages @@ -169,7 +182,6 @@ def read_conda_packages_from_dict(env_desc: dict) -> Tuple[List[str], List[str]] for package in conda_packages: if isinstance(package, dict) and "pip" in package: pip_packages = package["pip"] - # pip_packages = [parse_pip_requirement(line) for line in pip_packages] elif isinstance(package, str): conda_packages_.append(package) diff --git a/learnware/client/utils.py b/learnware/client/utils.py index 0e11ae0..132c9a0 100644 --- a/learnware/client/utils.py +++ b/learnware/client/utils.py @@ -10,24 +10,20 @@ logger = get_module_logger(module_name="client_utils") def system_execute(args, timeout=None, env=None, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE): - if env is None: - env = os.environ.copy() - pass - - if isinstance(args, str): - pass - else: - args = " ".join(args) - pass + env = os.environ.copy() if env is None else env + args = args if isinstance(args, str) else " ".join(args) com_process = subprocess.run(args, stdout=stdout, stderr=stderr, timeout=timeout, env=env, shell=True) try: com_process.check_returncode() except subprocess.CalledProcessError as err: - errmsg = com_process.stderr.decode() - logger.warning(f"System Execute Error: {errmsg}") - raise Exception(errmsg) + if err.stderr is not None: + errmsg = err.stderr.decode() + logger.warning(f"System Execute Error: {errmsg}") + raise err + + return com_process def remove_enviroment(conda_env): @@ -76,7 +72,7 @@ def install_environment(learnware_dirpath, conda_env): "-n", f"{conda_env}", "--no-capture-output", - "python3", + "python", "-m", "pip", "install", @@ -95,7 +91,7 @@ def install_environment(learnware_dirpath, conda_env): "-n", f"{conda_env}", "--no-capture-output", - "python3", + "python", "-m", "pip", "install", diff --git a/learnware/config.py b/learnware/config.py index 7fa89a4..71eeee7 100644 --- a/learnware/config.py +++ b/learnware/config.py @@ -165,7 +165,7 @@ _DEFAULT_CONFIG = { }, "database_url": f"sqlite:///{DATABASE_PATH}", "max_reduced_set_size": 1310720, - "backend_host": "http://www.lamda.nju.edu.cn/learnware/api", + "backend_host": "https://bmwu.cloud/api", "random_seed": 0, } diff --git a/learnware/market/easy/checker.py b/learnware/market/easy/checker.py index eb6fe75..7995d94 100644 --- a/learnware/market/easy/checker.py +++ b/learnware/market/easy/checker.py @@ -57,7 +57,7 @@ class EasySemanticChecker(BaseChecker): return EasySemanticChecker.NONUSABLE_LEARNWARE, "EasySemanticChecker Success" - except AssertionError as err: + except Exception as err: logger.warning(f"semantic_specification is not valid due to {err}!") return EasySemanticChecker.INVALID_LEARNWARE, traceback.format_exc() diff --git a/learnware/market/heterogeneous/organizer/__init__.py b/learnware/market/heterogeneous/organizer/__init__.py index 113b8c3..0258eea 100644 --- a/learnware/market/heterogeneous/organizer/__init__.py +++ b/learnware/market/heterogeneous/organizer/__init__.py @@ -96,7 +96,7 @@ class HeteroMapTableOrganizer(EasyOrganizer): ) if learnwere_status == BaseChecker.USABLE_LEARWARE and len(self._get_hetero_learnware_ids(learnware_id)): - self._update_learware_hetero_sepc(learnware_id) + self._update_learware_hetero_spec(learnware_id) if self.auto_update: self.count_down -= 1 @@ -113,7 +113,7 @@ class HeteroMapTableOrganizer(EasyOrganizer): f"Market mapping train completed. Now update HeteroMapTableSpecification for {training_learnware_ids}" ) self.market_mapping = updated_market_mapping - self._update_learware_hetero_sepc(training_learnware_ids) + self._update_learware_hetero_spec(training_learnware_ids) self.count_down = self.auto_update_limit @@ -167,7 +167,7 @@ class HeteroMapTableOrganizer(EasyOrganizer): """ final_status = super(HeteroMapTableOrganizer, self).update_learnware(id, zip_path, semantic_spec, check_status) if final_status == BaseChecker.USABLE_LEARWARE and len(self._get_hetero_learnware_ids(id)): - self._update_learware_hetero_sepc(id) + self._update_learware_hetero_spec(id) return final_status def _reload_learnware_hetero_spec(self, learnware_id): @@ -178,7 +178,7 @@ class HeteroMapTableOrganizer(EasyOrganizer): hetero_spec.load(hetero_spec_path) self.learnware_list[learnware_id].update_stat_spec(hetero_spec.type, hetero_spec) else: - self._update_learware_hetero_sepc(learnware_id) + self._update_learware_hetero_spec(learnware_id) logger.info(f"Reload HeteroMapTableSpecification for hetero spec {learnware_id} succeed!") except Exception as err: logger.error(f"Reload HeteroMapTableSpecification for hetero spec {learnware_id} failed! due to {err}.") @@ -196,7 +196,7 @@ class HeteroMapTableOrganizer(EasyOrganizer): if len(self._get_hetero_learnware_ids(learnware_id)): self._reload_learnware_hetero_spec(learnware_id) - def _update_learware_hetero_sepc(self, ids: Union[str, List[str]]): + def _update_learware_hetero_spec(self, ids: Union[str, List[str]]): """Update learnware by ids, attempting to generate HeteroMapTableSpecification for them. Parameters diff --git a/tests/test_hetero_market/test_hetero.py b/tests/test_hetero_market/test_hetero.py index be828e5..41b4261 100644 --- a/tests/test_hetero_market/test_hetero.py +++ b/tests/test_hetero_market/test_hetero.py @@ -35,6 +35,7 @@ user_semantic = { "Scenario": {"Values": ["Education"], "Type": "Tag"}, "Description": {"Values": "", "Type": "String"}, "Name": {"Values": "", "Type": "String"}, + "License": {"Values": ["MIT"], "Type": "Class"}, } @@ -259,15 +260,15 @@ class TestMarket(unittest.TestCase): str(key): semantic_spec["Input"]["Description"][str(key)] for key in range(user_dim) } user_info = BaseUserInfo(semantic_spec=semantic_spec, stat_info={"RKMETableSpecification": user_spec}) - + search_result = hetero_market.search_learnware(user_info) single_result = search_result.get_single_results() multiple_result = search_result.get_multiple_results() - + print(f"search result of user{idx}:") for single_item in single_result: print(f"score: {single_item.score}, learnware_id: {single_item.learnware.id}") - + for multiple_item in multiple_result: print( f"mixture_score: {multiple_item.score}, mixture_learnware_ids: {[item.id for item in multiple_item.learnwares]}" @@ -335,7 +336,7 @@ class TestMarket(unittest.TestCase): print(f"search result of user{idx}:") for single_item in single_result: print(f"score: {single_item.score}, learnware_id: {single_item.learnware.id}") - + for multiple_item in multiple_result: print(f"mixture_score: {multiple_item.score}\n") mixture_id = " ".join([learnware.id for learnware in multiple_item.learnwares]) @@ -363,9 +364,11 @@ class TestMarket(unittest.TestCase): # print search results for single_item in single_result: print(f"score: {single_item.score}, learnware_id: {single_item.learnware.id}") - + for multiple_item in multiple_result: - print(f"mixture_score: {multiple_item.score}, mixture_learnware_ids: {[item.id for item in multiple_item.learnwares]}") + print( + f"mixture_score: {multiple_item.score}, mixture_learnware_ids: {[item.id for item in multiple_item.learnwares]}" + ) # single model reuse hetero_learnware = HeteroMapAlignLearnware(single_result[0].learnware, mode="regression") diff --git a/tests/test_learnware_client/test_load_learnware.py b/tests/test_learnware_client/test_load_learnware.py index fafb261..0c20939 100644 --- a/tests/test_learnware_client/test_load_learnware.py +++ b/tests/test_learnware_client/test_load_learnware.py @@ -23,7 +23,7 @@ class TestLearnwareLoad(unittest.TestCase): def setUp(self): self.client = LearnwareClient() root = os.path.dirname(__file__) - self.learnware_ids = ["00000084", "00000154", "00000155"] + self.learnware_ids = ["00000910", "00000899", "00000900"] self.zip_paths = [os.path.join(root, x) for x in ["1.zip", "2.zip", "3.zip"]] def _test_load_learnware_by_zippath(self, runnable_option): diff --git a/tests/test_workflow/test_workflow.py b/tests/test_workflow/test_workflow.py index 0702a13..83f39ba 100644 --- a/tests/test_workflow/test_workflow.py +++ b/tests/test_workflow/test_workflow.py @@ -27,6 +27,7 @@ user_semantic = { "Scenario": {"Values": ["Education"], "Type": "Tag"}, "Description": {"Values": "", "Type": "String"}, "Name": {"Values": "", "Type": "String"}, + "License": {"Values": ["MIT"], "Type": "Class"}, } @@ -145,11 +146,15 @@ class TestWorkflow(unittest.TestCase): user_info = BaseUserInfo(semantic_spec=semantic_spec) search_result = easy_market.search_learnware(user_info) single_result = search_result.get_single_results() - + print("User info:", user_info.get_semantic_spec()) print(f"Search result:") for search_item in single_result: - print("Choose learnware:", search_item.learnware.id, search_item.learnware.get_specification().get_semantic_spec()) + print( + "Choose learnware:", + search_item.learnware.id, + search_item.learnware.get_specification().get_semantic_spec(), + ) rmtree(test_folder) # rm -r test_folder @@ -176,12 +181,12 @@ class TestWorkflow(unittest.TestCase): single_result = search_results.get_single_results() multiple_result = search_results.get_multiple_results() - + assert len(single_result) >= 1, f"Statistical search failed!" print(f"search result of user{idx}:") for search_item in single_result: print(f"score: {search_item.score}, learnware_id: {search_item.learnware.id}") - + for mixture_item in multiple_result: print(f"mixture_score: {mixture_item.score}\n") mixture_id = " ".join([learnware.id for learnware in mixture_item.learnwares]) @@ -229,8 +234,8 @@ class TestWorkflow(unittest.TestCase): def suite(): _suite = unittest.TestSuite() - #_suite.addTest(TestWorkflow("test_prepare_learnware_randomly")) - #_suite.addTest(TestWorkflow("test_upload_delete_learnware")) + # _suite.addTest(TestWorkflow("test_prepare_learnware_randomly")) + # _suite.addTest(TestWorkflow("test_upload_delete_learnware")) _suite.addTest(TestWorkflow("test_search_semantics")) _suite.addTest(TestWorkflow("test_stat_search")) _suite.addTest(TestWorkflow("test_learnware_reuse"))