cellects.core.program_organizer

cellects.core.program_organizer

This file contains the class constituting the link between the graphical interface and the computations. First, Cellects analyze one image in order to get a color space combination maximizing the contrast between the specimens and the background. Second, Cellects automatically delineate each arena. Third, Cellects write one video for each arena. Fourth, Cellects segments the video and apply post-processing algorithms to improve the segmentation. Fifth, Cellects extract variables and store them in .csv files.

ProgramOrganizer

Organizes and manages variables, configuration settings, and processing workflows for motion analysis in a Cellects project.

This class maintains global state and analysis-specific data structures, handles file operations, processes image/video inputs, and generates output tables. It provides methods to load/save configurations, segment images, track objects across frames, and export results with metadata.

Attributes:

Name	Type	Description
one_arena_done	bool	Flag indicating whether a single arena has been processed.
reduce_image_dim	bool	Whether image dimensions should be reduced (e.g., from color to grayscale).
first_exp_ready_to_run	bool	Indicates if the initial experiment setup is complete and ready for execution.
videos	OneVideoPerBlob or None	Video processing container instance.
motion	MotionAnalysis or None	Motion tracking and analysis module.
all	dict	Global configuration parameters for the entire workflow.
vars	dict	Analysis-specific variables used by MotionAnalysis.
first_im, last_im	ndarray or None	First and last images of the dataset for preprocessing.
data_list	list of str	List of video/image file paths in the working directory.
computed_video_options	np.ndarray of bool	Flags indicating which video processing options have been applied.
one_row_per_arena, one_row_per_frame	DataFrame or None	Result tables for different levels of analysis (per arena, per frame, and oscillating clusters).

Methods:

save_variable_dict() : Save configuration dictionaries to file. load_variable_dict() : Load saved configuration or initialize defaults. look_for_data() : Discover video/image files in the working directory. update_folder_id(...) : Update folder-specific metadata based on file structure. ...

Source code in src/cellects/core/program_organizer.py

class ProgramOrganizer:
    """
    Organizes and manages variables, configuration settings, and processing workflows for motion analysis in a Cellects project.

    This class maintains global state and analysis-specific data structures, handles file operations,
    processes image/video inputs, and generates output tables. It provides methods to load/save configurations,
    segment images, track objects across frames, and export results with metadata.

    Attributes
    ----------
    one_arena_done : bool
        Flag indicating whether a single arena has been processed.
    reduce_image_dim : bool
        Whether image dimensions should be reduced (e.g., from color to grayscale).
    first_exp_ready_to_run : bool
        Indicates if the initial experiment setup is complete and ready for execution.
    videos : OneVideoPerBlob or None
        Video processing container instance.
    motion : MotionAnalysis or None
        Motion tracking and analysis module.
    all : dict
        Global configuration parameters for the entire workflow.
    vars : dict
        Analysis-specific variables used by `MotionAnalysis`.
    first_im, last_im : np.ndarray or None
        First and last images of the dataset for preprocessing.
    data_list : list of str
        List of video/image file paths in the working directory.
    computed_video_options : np.ndarray of bool
        Flags indicating which video processing options have been applied.
    one_row_per_arena, one_row_per_frame : pd.DataFrame or None
        Result tables for different levels of analysis (per arena, per frame, and oscillating clusters).

    Methods:
    --------
    save_variable_dict() : Save configuration dictionaries to file.
    load_variable_dict() : Load saved configuration or initialize defaults.
    look_for_data() : Discover video/image files in the working directory.
    update_folder_id(...) : Update folder-specific metadata based on file structure.
    ...

    """
    def __init__(self):
        """
            This class stores all variables required for analysis as well as
            methods to process it.
            Global variables (i.e. that does not concern the MotionAnalysis)
            are directly stored in self.
            Variables used in the MotionAnalysis class are stored in a dict
            called self.vars
        """
        self.one_arena_done: bool = False
        self.reduce_image_dim: bool = False
        self.first_exp_ready_to_run: bool = False
        self.sample_number = 1
        self.top = None
        self.motion = None
        self.analysis_instance = None
        self.computed_video_options = np.zeros(5, bool)
        self.vars = {}
        self.all = {}
        self.all['folder_list'] = []
        self.vars['first_detection_frame'] = 0
        self.first_im = None
        self.last_im = None
        self.starting_blob_hsize_in_pixels = None
        self.vars['first_move_threshold'] = 10
        self.vars['convert_for_origin'] = None
        self.vars['convert_for_motion'] = None
        self.current_combination_id = 0
        self.data_list = []
        self.one_row_per_arena = None
        self.one_row_per_frame = None
        self.not_analyzed_individuals = None
        self.bio_mask = None
        self.back_mask = None
        self.visualize: bool = True
        self.network_shaped: bool = False
        self.update_background_luminosity: bool = False

    def update_variable_dict(self):
        """

        Update the `all` and `vars` dictionaries with new data from `DefaultDicts`.

        This method updates the `all` and `vars` dictionaries of the current object with
        data from a new instance of `DefaultDicts`. It checks if any keys or descriptors
        are missing and adds them accordingly.

        Examples
        --------
        >>> organizer = ProgramOrganizer()
        >>> organizer.update_variable_dict()
        """
        dd = DefaultDicts()
        all = len(dd.all) != len(self.all)
        vars = len(dd.vars) != len(self.vars)
        all_desc = not 'descriptors' in self.all or len(dd.all['descriptors']) != len(self.all['descriptors'])
        vars_desc = not 'descriptors' in self.vars or len(dd.vars['descriptors']) != len(self.vars['descriptors'])
        if all:
            for key, val in dd.all.items():
                if not key in self.all:
                    self.all[key] = val
        if vars:
            for key, val in dd.vars.items():
                if not key in self.vars:
                    self.vars[key] = val
        if all_desc:
            for key, val in dd.all['descriptors'].items():
                if not key in self.all['descriptors']:
                    self.all['descriptors'][key] = val
        if vars_desc:
            for key, val in dd.vars['descriptors'].items():
                if not key in self.vars['descriptors']:
                    self.vars['descriptors'][key] = val
        self._set_analyzed_individuals()

    def save_variable_dict(self):
        """
        Saves the configuration dictionaries (`self.all` and `self.vars`) to a json file.

        If bio_mask or back_mask are not required for all folders, they are excluded from the saved data.

        Notes
        -----
        This method is used to preserve state between Cellects sessions or restart scenarios.
        """
        logging.info("Update -cellects_settings.json- in the Cellects folder")
        all_vars = self.all.copy()
        all_vars['vars'] = self.vars.copy()
        all_vars['vars'].pop('crop_coord', None)
        all_vars['vars'].pop('arenas_coord', None)
        all_vars['vars'].pop('exif', None)
        all_vars.pop('initial_bio_mask', None)
        all_vars.pop('initial_back_mask', None)
        write_json(ALL_VARS_JSON_FILE, all_vars)

    def save_first_image(self):
        """
        Save the first image's validated shapes to an HDF5 file.

        If the current combination ID is valid and has a non-empty set of
        image combinations, save the validated shapes to 'cellects_data.h5'.

        Notes
        -----
        This function assumes that `self.first_image` and its attributes are already defined.
        It uses the smallest memory-efficient array from `np.nonzero(validated_shapes)` to save space.
        """
        if self.first_image is not None and self.first_image.im_combinations is not None and len(self.first_image.im_combinations) > 0:
            validated_shapes = self.first_image.im_combinations[self.current_combination_id]['binary_image']
            write_h5('cellects_data.h5', smallest_memory_array(np.nonzero(validated_shapes)), 'validated_shapes')

    def save_masks(self, remove_unused_masks: bool = True):
        """
        Conditionally save or remove masks to disk for batch processing (several folders).

        When analyzing several folders, the same masks are (optionally) saved to ease the first image detection.
        After user input, unused masks should be removed while at other times,
        calling this method should not remove that information.


        Parameters
        ----------
        remove_unused_masks : bool, optional
            If True and there is no user-made mask, remove saved masks from disk.
            Default is True.

        Notes
        -----
        This function saves the masks to an HDF5 file saved in the config folder (to be accessible anywhere)
        """
        if self.all['keep_cell_and_back_for_all_folders']:
            if self.bio_mask is not None:
                write_h5(CONFIG_DIR / 'masks.h5', self.bio_mask, 'initial_bio_mask')
            if self.back_mask is not None:
                write_h5(CONFIG_DIR / 'masks.h5', self.back_mask, 'initial_back_mask')
            if remove_unused_masks:
                if self.back_mask is None:
                    remove_h5_key(CONFIG_DIR / 'masks.h5', 'initial_back_mask')
                if self.bio_mask is None:
                    remove_h5_key(CONFIG_DIR / 'masks.h5', 'initial_bio_mask')
        else:
            self.all.pop('initial_bio_mask', None)
            self.all.pop('initial_back_mask', None)
            if os.path.isfile(CONFIG_DIR / 'masks.h5'):
                os.remove(CONFIG_DIR / 'masks.h5')


    def load_variable_dict(self):
        """
        Loads configuration dictionaries from a pickle file if available, otherwise initializes defaults.

        Tries to load saved parameters. If the file doesn't exist or loading fails due to corruption,
        default values are used instead (logging relevant warnings).

        Raises
        ------
        FileNotFoundError
            If no valid configuration file is found and default initialization fails.

        Notes
        -----
        This method ensures robust operation by handling missing or corrupted configuration files gracefully.
        """
        if os.path.isfile(ALL_VARS_JSON_FILE):
            logging.info("Load the parameters from cellects_settings.json in the config of the Cellects folder")
            try:
                self.all = read_json(ALL_VARS_JSON_FILE)
                self.vars = self.all['vars']
                self.update_variable_dict()
                logging.info("Success to load the parameters dictionaries from the Cellects folder")
            except Exception as exc:
                logging.error(f"Initialize default parameters because error: {exc}")
                default_dicts = DefaultDicts()
                self.all = default_dicts.all
                self.vars = default_dicts.vars
        else:
            logging.info("Initialize default parameters")
            default_dicts = DefaultDicts()
            self.all = default_dicts.all
            self.vars = default_dicts.vars
        if self.all['cores'] == 1:
            self.all['cores'] = os.cpu_count() - 1

    def look_for_data(self):
        """
        Discovers all relevant video/image data in the working directory.

        Uses natural sorting to handle filenames with numeric suffixes. Validates file consistency and logs warnings
        if filename patterns are inconsistent across folders.

        Raises
        ------
        ValueError
            If no files match the specified naming convention.

        Notes
        -----
        This method assumes all data files follow a predictable pattern with numeric extensions. Use caution in
        unpredictable directory structures where this may fail silently or produce incorrect results.

        Examples
        --------
        >>> organizer.look_for_data()
        >>> print(organizer.data_list)
        ['/path/to/video1.avi', '/path/to/video2.avi']
        """
        os.chdir(Path(self.all['global_pathway']))
        logging.info(f"Dir: {self.all['global_pathway']}")
        self.data_list = insensitive_glob(self.all['radical'] + '*' + self.all['extension'])  # Provides a list ordered by last modification date
        self.all['folder_list'] = []
        self.all['folder_number'] = 1
        self.vars['first_detection_frame'] = 0
        if len(self.data_list) > 0:
            self._sort_data_list()
            self.sample_number = self.all['first_folder_sample_number']
        else:
            content = os.listdir()
            for obj in content:
                if not os.path.isfile(obj):
                    data_list = insensitive_glob(obj + "/" + self.all['radical'] + '*' + self.all['extension'])
                    if len(data_list) > 0:
                        self.all['folder_list'].append(obj)
                        self.all['folder_number'] += 1
            self.all['folder_list'] = np.sort(self.all['folder_list']).tolist()

            if isinstance(self.all['sample_number_per_folder'], int) or len(self.all['sample_number_per_folder']) == 1:
                self.all['sample_number_per_folder'] = np.repeat(self.all['sample_number_per_folder'], self.all['folder_number']).tolist()

    def _sort_data_list(self):
        """
        Sorts the data list using natural sorting.

        Extended Description
        --------------------
        This function sorts the `data_list` attribute of an instance using the natsort library,
        which is useful when filenames have a mixture of numbers and letters.
        """
        if len(self.data_list) > 0:
            lengths = vectorized_len(self.data_list)
            if len(lengths) > 1 and np.max(np.diff(lengths)) > np.log10(len(self.data_list)):
                logging.error(f"File names present strong variations and cannot be correctly sorted.")
            if self.all['im_or_vid'] == 1:
                wrong_files = np.nonzero(np.char.startswith(self.data_list, "ind_", ))[0]
            else:
                wrong_files = np.nonzero(np.char.startswith(self.data_list, "Analysis efficiency, ", ))[0]
            for w_im in wrong_files[::-1]:
                self.data_list.pop(w_im)
            self.data_list = natsort.natsorted(self.data_list)
        if self.all['im_or_vid'] == 1:
            self.vars['video_list'] = self.data_list
        else:
            self.vars['video_list'] = None

    def update_folder_id(self, sample_number: int, folder_name: str=""):
        """
        Update the current working directory and data list based on the given sample number
        and optional folder name.

        Parameters
        ----------
        sample_number : int
            The number of samples to analyze.
        folder_name : str, optional
            The name of the folder to change to. Default is an empty string.

        Notes
        -----
        This function changes the current working directory to the specified folder name
        and updates the data list based on the file names in that directory. It also performs
        sorting of the data list and checks for strong variations in file names.

        """
        os.chdir(Path(self.all['global_pathway']) / folder_name)
        self.data_list = insensitive_glob(
            self.all['radical'] + '*' + self.all['extension'])  # Provides a list ordered by last modification date
        # Sorting is necessary when some modifications (like rotation) modified the last modification date
        self._sort_data_list()
        if self.all['im_or_vid'] == 1:
            self.sample_number = sample_number
        else:
            self.vars['img_number'] = len(self.data_list)
            self.sample_number = sample_number
        if not 'analyzed_individuals' in self.vars:
            self._set_analyzed_individuals()

    def _set_analyzed_individuals(self):
        """
        Set the analyzed individuals variable in the dataset.
        """
        self.vars['analyzed_individuals'] = list(range(1, self.sample_number + 1))
        if self.not_analyzed_individuals is not None:
            for ind in self.not_analyzed_individuals:
                self.vars['analyzed_individuals'].remove(ind)

    def load_data_to_run_cellects_quickly(self):
        """
        Load data from a pickle file and update the current state of the object.

        Summarizes, loads, and validates data needed to run Cellects,
        updating the object's state accordingly. If the necessary data
        are not present or valid, it ensures the experiment is marked as
        not ready to run.

        Parameters
        ----------
        self : CellectsObject
            The instance of the class (assumed to be a subclass of
            CellectsObject) that this method belongs to.

        Returns
        -------
        None

        Notes
        -----
        This function relies on the presence of a pickle file 'cellects_settings.json'.
        It updates the state of various attributes based on the loaded data
        and logs appropriate messages.
        """
        self.analysis_instance = None
        self.first_im = None
        self.first_image = None
        self.last_image = None
        current_global_pathway = self.all['global_pathway']
        folder_number = self.all['folder_number']
        if folder_number > 1:
            folder_list = self.all['folder_list'].copy()
            sample_number_per_folder = self.all['sample_number_per_folder'].copy()

        self.first_exp_ready_to_run: bool = False
        if os.path.isfile('cellects_settings.json'):
            data_to_run_cellects_quickly = read_json('cellects_settings.json')
            if data_to_run_cellects_quickly is None:
                data_to_run_cellects_quickly = {}
            if (os.path.isfile('ind_1.h5')) and (os.path.isfile('cellects_data.h5')) and ('all' in data_to_run_cellects_quickly):
                ind1_keys = get_h5_keys('ind_1.h5')
                cellects_data_keys = get_h5_keys('cellects_data.h5')
                if 'origin_coord' in ind1_keys and 'arenas_coord' in cellects_data_keys and 'exif' in cellects_data_keys:
                    logging.info("Success to load cellects_settings.json from the user chosen directory")
                    self.all = data_to_run_cellects_quickly['all']
                    # If you want to add a new variable, first run an updated version of all_vars_dict,
                    # then put a breakpoint here and run the following + self.save_data_to_run_cellects_quickly() :
                    self.vars = self.all['vars']
                    self.update_variable_dict()
                    folder_changed = False
                    if current_global_pathway != self.all['global_pathway']:
                        folder_changed = True
                        logging.info("Although the folder is ready, it is not at the same place as it was during creation, updating")
                        self.all['global_pathway'] = current_global_pathway
                    if folder_number > 1:
                        self.all['global_pathway'] = current_global_pathway
                        self.all['folder_list'] = folder_list
                        self.all['folder_number'] = folder_number
                        self.all['sample_number_per_folder'] = sample_number_per_folder
                        self.all['first_folder_sample_number'] = sample_number_per_folder[0]

                    if len(self.data_list) == 0:
                        self.look_for_data()
                        if folder_changed and folder_number > 1 and len(self.all['folder_list']) > 0:
                            self.update_folder_id(self.all['sample_number_per_folder'][0], self.all['folder_list'][0])
                    if len(self.data_list) > 0:
                        self.get_first_image()
                        self.get_last_image()
                        self.top, self.bot, self.left, self.right = read_h5('cellects_data.h5', 'arenas_coord')
                        self.vars['arenas_coord'] = [self.top, self.bot, self.left, self.right]
                        self.vars['exif'] = read_h5('cellects_data.h5', 'exif')
                        self.vars['crop_coord'] = None
                        if self.all['automatically_crop'] and 'crop_coord' in cellects_data_keys:
                            ccy1, ccy2, ccx1, ccx2 = read_h5('cellects_data.h5', 'crop_coord')
                            self.first_image.crop_coord = [ccy1, ccy2, ccx1, ccx2]
                            self.vars['crop_coord'] = self.first_image.crop_coord
                            logging.info("Crop first image")
                            self.first_image.automatically_crop(self.first_image.crop_coord)
                            logging.info("Crop last image")
                            self.last_image.automatically_crop(self.first_image.crop_coord)
                        shapes_coord = read_h5('cellects_data.h5','validated_shapes')
                        if shapes_coord is not None:
                            self.first_image.validated_shapes = np.zeros(self.first_image.image.shape[:2], np.uint8)
                            self.first_image.validated_shapes[shapes_coord[0], shapes_coord[1]] = 1
                            self.first_image.im_combinations = []
                            self.current_combination_id = 0
                            self.first_image.im_combinations.append({})
                            self.first_image.im_combinations[self.current_combination_id]['csc'] = self.vars['convert_for_origin']
                            self.first_image.im_combinations[self.current_combination_id]['binary_image'] = self.first_image.validated_shapes
                            self.first_image.im_combinations[self.current_combination_id]['shape_number'] = data_to_run_cellects_quickly['shape_number']
                            if not 'average_pixel_size' in self.vars:
                                self.get_average_pixel_size()
                            if not 'lighter_background' in self.vars:
                                self.find_if_lighter_background()
                            background = read_h5(f'ind_{1}.h5', 'background')
                            if not self.vars['subtract_background'] or (self.vars['subtract_background'] and background is not None):
                                self.first_exp_ready_to_run = True
        if self.first_exp_ready_to_run:
            logging.info("The current folder is ready to run")
        else:
            logging.info("The current folder is not ready to run")

    def save_data_to_run_cellects_quickly(self, new_one_if_does_not_exist: bool=True):
        """
        Save data to a pickled file if it does not exist or update existing data.

        Parameters
        ----------
        new_one_if_does_not_exist : bool, optional
            Whether to create a new data file if it does not already exist.
            Default is True.

        Notes
        -----
        This method logs various information about its operations and handles the writing of data to a pickled file.
        """
        data_to_run_cellects_quickly = None
        if os.path.isfile('cellects_settings.json'):
            logging.info("Update -cellects_settings.json- in the user chosen directory")
            data_to_run_cellects_quickly = read_json('cellects_settings.json')
            if data_to_run_cellects_quickly is None:
                os.remove('cellects_settings.json')
                logging.error("Failed to load cellects_settings.json before update. Remove pre existing.")
        else:
            if new_one_if_does_not_exist:
                logging.info("Create cellects_settings.json in the user chosen directory")
                data_to_run_cellects_quickly = {}
        if data_to_run_cellects_quickly is not None:
            if self.first_image is not None and self.first_image.im_combinations is not None and len(self.first_image.im_combinations) > 0:
                data_to_run_cellects_quickly['shape_number'] = self.first_image.im_combinations[self.current_combination_id]['shape_number']
            all_vars = self.all.copy()
            all_vars['vars'] = self.vars.copy()
            all_vars['vars'].pop('crop_coord', None)
            all_vars['vars'].pop('arenas_coord', None)
            all_vars['vars'].pop('exif', None)
            all_vars.pop('initial_bio_mask', None)
            all_vars.pop('initial_back_mask', None)
            data_to_run_cellects_quickly['all'] = all_vars
            write_json('cellects_settings.json', data_to_run_cellects_quickly)

    def save_coordinates(self):
        """
        Summarize the coordinates of images and video.

        Combine the crop coordinates from the first image with additional
        coordinates for left, right, top, and bottom boundaries to form a list of
        video coordinates. If the crop coordinates are not already set, initialize
        them to cover the entire image.

        Returns
        -------
        list of int
            A list containing the coordinates [left, right, top, bottom] for video.

        """
        if self.first_image.crop_coord is None:
            self.first_image.crop_coord = [0, self.first_image.image.shape[0], 0, self.first_image.image.shape[1]]
        if isinstance(self.top, np.ndarray):
            arenas_coord = [self.top.tolist(), self.bot.tolist(),self.left.tolist(), self.right.tolist()]
        else:
            arenas_coord = [self.top, self.bot,self.left, self.right]
        self.vars['crop_coord'] = self.first_image.crop_coord
        self.vars['arenas_coord'] = arenas_coord
        write_h5('cellects_data.h5', self.vars['crop_coord'], 'crop_coord')
        write_h5('cellects_data.h5', self.vars['arenas_coord'], 'arenas_coord')
        self.all['overwrite_unaltered_videos'] = True

    def get_first_image(self, first_im: NDArray=None, sample_number: int=None):
        """
        Load and process the first image or frame from a video.

        This method handles loading the first image or the first frame of a video
        depending on whether the data is an image or a video. It performs necessary
        preprocessing and initializes relevant attributes for subsequent analysis.
        """
        if sample_number is not None:
            self.sample_number = sample_number
        self.reduce_image_dim = False
        if first_im is not None:
            self.first_im = first_im
        else:
            logging.info("Load first image")
            if self.all['im_or_vid'] == 1:
                if self.analysis_instance is None:
                    self.analysis_instance = video2numpy(self.data_list[0])
                    self.sample_number = len(self.data_list)
                    self.vars['img_number'] = self.analysis_instance.shape[0]
                    self.first_im = self.analysis_instance[0, ...]
                    self.vars['dims'] = self.analysis_instance.shape[:3]
                else:
                    self.first_im = self.analysis_instance[self.vars['first_detection_frame'], ...]

            else:
                self.vars['img_number'] = len(self.data_list)
                self.all['raw_images'] = is_raw_image(self.data_list[0])
                self.first_im = readim(self.data_list[self.vars['first_detection_frame']], self.all['raw_images'])
                self.vars['dims'] = [self.vars['img_number'], self.first_im.shape[0], self.first_im.shape[1]]

                if len(self.first_im.shape) == 3:
                    if np.all(np.equal(self.first_im[:, :, 0], self.first_im[:, :, 1])) and np.all(
                            np.equal(self.first_im[:, :, 1], self.first_im[:, :, 2])):
                        self.reduce_image_dim = True
                    if self.reduce_image_dim:
                        self.first_im = self.first_im[:, :, 0]

        self.first_image = OneImageAnalysis(self.first_im, self.sample_number)
        self.vars['already_greyscale'] = self.first_image.already_greyscale
        if self.vars['already_greyscale']:
            self.vars["convert_for_origin"] = {"bgr": [1, 1, 1], "logical": "None"}
            self.vars["convert_for_motion"] = {"bgr": [1, 1, 1], "logical": "None"}
        if np.mean((np.mean(self.first_image.image[2, :, ...]), np.mean(self.first_image.image[-3, :, ...]), np.mean(self.first_image.image[:, 2, ...]), np.mean(self.first_image.image[:, -3, ...]))) > 127:
            self.vars['contour_color']: np.uint8 = 0
        else:
            self.vars['contour_color']: np.uint8 = 255
        if self.vars['first_detection_frame'] > 0:
            self.vars['origin_state'] = 'invisible'

    def load_masks(self):
        """"""
        if self.all['keep_cell_and_back_for_all_folders']:
            self.bio_mask = read_h5(CONFIG_DIR / 'masks.h5', 'initial_bio_mask')
            self.back_mask = read_h5(CONFIG_DIR / 'masks.h5', 'initial_back_mask')

    def get_last_image(self, last_im: NDArray=None):
        """

        Load the last image from a video or image list and process it based on given parameters.

        Parameters
        ----------
        last_im : NDArray, optional
            The last image to be loaded. If not provided, the last image will be loaded from the data list.
        """
        logging.info("Load last image")
        if last_im is not None:
            self.last_im = last_im
        else:
            if self.all['im_or_vid'] == 1:
                self.last_im = self.analysis_instance[-1, ...]
            else:
                is_landscape = self.first_image.image.shape[0] < self.first_image.image.shape[1]
                self.last_im = read_and_rotate(self.data_list[-1], self.first_im, self.all['raw_images'], is_landscape)
                if self.reduce_image_dim:
                    self.last_im = self.last_im[:, :, 0]
        self.last_image = OneImageAnalysis(self.last_im)

    def save_exif(self):
        """
        Extract EXIF data from image or video files.

        Notes
        -----
        If `extract_time_interval` is True and unsuccessful, arbitrary time steps will be used.
        Timings are normalized to minutes for consistency across different files.
        """
        self.vars['time_step_is_arbitrary'] = True
        if self.all['im_or_vid'] == 1:
            if not 'dims' in self.vars:
                self.vars['dims'] = self.analysis_instance.shape[:3]
            timings = np.arange(self.vars['dims'][0])
        else:
            timings = np.arange(len(self.data_list))
            if sys.platform.startswith('win'):
                pathway = os.getcwd() + '\\'
            else:
                pathway = os.getcwd() + '/'
            if not 'extract_time_interval' in self.all:
                self.all['extract_time_interval'] = True
            if self.all['extract_time_interval']:
                self.vars['time_step'] = 1
                try:
                    timings = extract_time(pathway, self.data_list, self.all['raw_images'])
                    timings = timings - timings[0]
                    timings = timings / 60
                    time_step = np.diff(timings)
                    if len(time_step) > 0:
                        time_step = np.mean(time_step)
                        digit_nb = 0
                        for i in str(time_step):
                            if i in {'.'}:
                                pass
                            elif i in {'0'}:
                                digit_nb += 1
                            else:
                                break
                        self.vars['time_step'] = round(time_step, digit_nb + 1)
                        self.vars['time_step_is_arbitrary'] = False
                except:
                    pass
            else:
                timings = np.arange(0, len(self.data_list) * self.vars['time_step'], self.vars['time_step'])
                self.vars['time_step_is_arbitrary'] = False
        self.vars['exif'] = timings.tolist()
        write_h5('cellects_data.h5', self.vars['exif'], 'exif')

    def fast_first_image_segmentation(self):
        """
        Segment the first or subsequent image in a series for biological and background masks.

        Notes
        -----
        This function processes the first or subsequent image in a sequence, applying biological and background masks,
        segmenting the image, and updating internal data structures accordingly. The function is specific to handling
        image sequences for biological analysis

        """
        if not "color_number" in self.vars:
            self.update_variable_dict()
        if self.vars['convert_for_origin'] is None:
            self.vars['convert_for_origin'] = {"logical": 'None', "PCA": [1, 1, 1]}
        self.first_image.convert_and_segment(self.vars['convert_for_origin'], self.vars["color_number"],
                                             self.all['initial_bio_mask'], self.all['initial_back_mask'], subtract_background=None,
                                             subtract_background2=None,
                                             rolling_window_segmentation=self.vars["rolling_window_segmentation"],
                                             filter_spec=self.vars["filter_spec"])
        if not self.first_image.drift_correction_already_adjusted:
            self.vars['drift_already_corrected'] = self.first_image.check_if_image_border_attest_drift_correction()
            if self.vars['drift_already_corrected']:
                logging.info("Cellects detected that the images have already been corrected for drift")
                self.first_image.convert_and_segment(self.vars['convert_for_origin'], self.vars["color_number"],
                                                     self.all['initial_bio_mask'], self.all['initial_back_mask'],
                                                     subtract_background=None, subtract_background2=None,
                                                     rolling_window_segmentation=self.vars["rolling_window_segmentation"],
                                                     filter_spec=self.vars["filter_spec"],
                                                     allowed_window=self.first_image.drift_mask_coord)

        shapes_features = shape_selection(self.first_image.binary_image, true_shape_number=self.sample_number,
                                          horizontal_size=self.starting_blob_hsize_in_pixels,
                                          spot_shape=self.all['starting_blob_shape'],
                                          several_blob_per_arena=self.vars['several_blob_per_arena'],
                                          bio_mask=self.all['initial_bio_mask'], back_mask=self.all['initial_back_mask'])
        self.first_image.validated_shapes, shape_number, stats, centroids = shapes_features
        self.first_image.shape_number = shape_number
        if self.first_image.im_combinations is None:
            self.first_image.im_combinations = []
        if len(self.first_image.im_combinations) == 0:
            self.first_image.im_combinations.append({})
        self.current_combination_id = np.min((self.current_combination_id, len(self.first_image.im_combinations) - 1))
        self.first_image.im_combinations[self.current_combination_id]['csc'] = self.vars['convert_for_origin']
        self.first_image.im_combinations[self.current_combination_id]['binary_image'] = self.first_image.validated_shapes
        if self.first_image.greyscale is not None:
            greyscale = self.first_image.greyscale
        else:
            greyscale = self.first_image.image
        self.first_image.im_combinations[self.current_combination_id]['converted_image'] = bracket_to_uint8_image_contrast(greyscale)
        self.first_image.im_combinations[self.current_combination_id]['shape_number'] = shape_number

    def fast_last_image_segmentation(self, bio_mask: NDArray[np.uint8] = None, back_mask: NDArray[np.uint8] = None):
        """
        Segment the first or subsequent image in a series for biological and background masks.

        Parameters
        ----------
        bio_mask : NDArray[np.uint8], optional
            The biological mask to be applied to the image.
        back_mask : NDArray[np.uint8], optional
            The background mask to be applied to the image.

        Returns
        -------
        None

        Notes
        -----
        This function processes the first or subsequent image in a sequence, applying biological and background masks,
        segmenting the image, and updating internal data structures accordingly. The function is specific to handling
        image sequences for biological analysis

        """
        if self.vars['convert_for_motion'] is None:
            self.vars['convert_for_motion'] = {"logical": 'None', "PCA": [1, 1, 1]}
        self.cropping(is_first_image=False)
        self.last_image.convert_and_segment(self.vars['convert_for_motion'], self.vars["color_number"],
                                            bio_mask, back_mask, self.first_image.subtract_background,
                                            self.first_image.subtract_background2,
                                            rolling_window_segmentation=self.vars["rolling_window_segmentation"],
                                            filter_spec=self.vars["filter_spec"])
        if self.vars['drift_already_corrected'] and not self.last_image.drift_correction_already_adjusted and not self.vars["rolling_window_segmentation"]['do']:
            self.last_image.check_if_image_border_attest_drift_correction()
            self.last_image.convert_and_segment(self.vars['convert_for_motion'], self.vars["color_number"],
                                                bio_mask, back_mask, self.first_image.subtract_background,
                                                self.first_image.subtract_background2,
                                                allowed_window=self.last_image.drift_mask_coord,
                                                filter_spec=self.vars["filter_spec"])

        if self.last_image.im_combinations is None:
            self.last_image.im_combinations = []
        if len(self.last_image.im_combinations) == 0:
            self.last_image.im_combinations.append({})
        self.current_combination_id = np.min((self.current_combination_id, len(self.last_image.im_combinations) - 1))
        self.last_image.im_combinations[self.current_combination_id]['csc'] = self.vars['convert_for_motion']
        self.last_image.im_combinations[self.current_combination_id]['binary_image'] = self.last_image.binary_image
        if self.last_image.greyscale is not None:
            greyscale = self.last_image.greyscale
        else:
            greyscale = self.last_image.image
        self.last_image.im_combinations[self.current_combination_id]['converted_image'] = bracket_to_uint8_image_contrast(greyscale)

    def pre_save_user_masks(self, bio_mask=None, back_mask=None):
        self.all['initial_bio_mask'] = None
        self.all['initial_back_mask'] = None
        if bio_mask is not None and bio_mask.any():
            self.all['initial_bio_mask'] = np.nonzero(bio_mask)
        if back_mask is not None and back_mask.any():
            self.all['initial_back_mask'] = np.nonzero(back_mask)

    def full_first_image_segmentation(self, first_param_known: bool, bio_mask: NDArray[np.uint8] = None, back_mask: NDArray[np.uint8] = None):
        shape_nb = 1
        if bio_mask is not None and bio_mask.any():
            shape_nb, ordered_image = cv2.connectedComponents((bio_mask > 0).astype(np.uint8))
            shape_nb -= 1
        self.pre_save_user_masks(bio_mask=bio_mask, back_mask=back_mask)
        if self.visualize:
            if not first_param_known and self.all['scale_with_image_or_cells'] == 0 and self.all["set_spot_size"]:
                self.get_average_pixel_size()
            else:
                self.starting_blob_hsize_in_pixels = None
            self.fast_first_image_segmentation()
            if not self.vars['several_blob_per_arena'] and self.all['initial_bio_mask'] is not None and shape_nb == self.sample_number and self.first_image.im_combinations[self.current_combination_id]['shape_number'] != self.sample_number:
                self.first_image.im_combinations[self.current_combination_id]['shape_number'] = shape_nb
                self.first_image.shape_number = shape_nb
                self.first_image.validated_shapes = (ordered_image > 0).astype(np.uint8)
                self.first_image.im_combinations[self.current_combination_id]['binary_image'] = self.first_image.validated_shapes
        else:
            params = init_params()
            params['is_first_image'] = True
            params['blob_nb'] = self.sample_number
            if self.vars["color_number"] > 2:
                params['kmeans_clust_nb'] = self.vars["color_number"]
            params['bio_mask'] = self.all['initial_bio_mask']
            params['back_mask'] = self.all['initial_back_mask']
            params['filter_spec'] = self.vars["filter_spec"]

            if first_param_known:
                if self.all['scale_with_image_or_cells'] == 0:
                    self.get_average_pixel_size()
                else:
                    self.starting_blob_hsize_in_pixels = None
                params['several_blob_per_arena'] = self.vars['several_blob_per_arena']
                params['blob_shape'] = self.all['starting_blob_shape']
                params['blob_size'] = self.starting_blob_hsize_in_pixels

            if len(self.first_im.shape) == 2:
                self.first_image.find_potential_filters(params)
            else:
                self.first_image.find_color_space_combinations(params)

    def full_last_image_segmentation(self, bio_mask: NDArray[np.uint8] = None, back_mask: NDArray[np.uint8] = None):
        if bio_mask is not None and bio_mask.any():
            bio_mask = np.nonzero(bio_mask)
        else:
            bio_mask = None
        if back_mask is not None and back_mask.any():
            back_mask = np.nonzero(back_mask)
        else:
            back_mask = None
        if self.last_im is None:
            self.get_last_image()
        self.cropping(False)
        self.get_background_to_subtract()
        if self.visualize:
            self.fast_last_image_segmentation(bio_mask=bio_mask, back_mask=back_mask)
        else:
            arenas_mask = None
            if self.all['are_gravity_centers_moving'] != 1:
                cr = [self.top, self.bot, self.left, self.right]
                arenas_mask = np.zeros_like(self.first_image.validated_shapes)
                for _i in np.arange(len(self.vars['analyzed_individuals'])):
                    if self.vars['arena_shape'] == 'circle':
                        ellipse = create_ellipse(cr[1][_i] - cr[0][_i], cr[3][_i] - cr[2][_i])
                        arenas_mask[cr[0][_i]: cr[1][_i], cr[2][_i]:cr[3][_i]] = ellipse
                    else:
                        arenas_mask[cr[0][_i]: cr[1][_i], cr[2][_i]:cr[3][_i]] = 1
            if self.network_shaped:
                self.last_image.network_detection(arenas_mask, csc_dict=self.vars["convert_for_motion"], lighter_background=None, bio_mask=bio_mask, back_mask=back_mask)
            else:
                ref_image = self.first_image.validated_shapes
                params = init_params()
                params['is_first_image'] = False
                params['several_blob_per_arena'] = self.vars['several_blob_per_arena']
                params['blob_nb'] = self.sample_number
                params['arenas_mask'] = arenas_mask
                params['ref_image'] = ref_image
                params['subtract_background'] = self.first_image.subtract_background
                params['bio_mask'] = bio_mask
                params['back_mask'] = back_mask
                params['filter_spec'] = self.vars["filter_spec"]
                if len(self.last_image.image.shape) == 2:
                    self.last_image.find_potential_filters(params)
                else:
                    self.last_image.find_color_space_combinations(params)

    def cropping(self, is_first_image: bool):
        """
        Crops the image based on specified conditions and settings.

        This method checks if drift correction has already been applied.
        If the image is the first one and hasn't been cropped yet, it will attempt
        to use pre-stored coordinates or compute new crop coordinates. If automatic
        cropping is enabled, it will apply the cropping process.

        Parameters
        ----------
        is_first_image : bool
            Indicates whether the image being processed is the first one in the sequence.
        """
        if not self.vars['drift_already_corrected'] and self.all['automatically_crop']:
            if is_first_image:
                if not self.first_image.cropped:
                    self.first_image.get_crop_coordinates()
                    self.first_image.automatically_crop(self.first_image.crop_coord)
            else:
                if not self.last_image.cropped:
                    self.last_image.automatically_crop(self.first_image.crop_coord)

    def get_average_pixel_size(self):
        """
        Calculate the average pixel size and related variables.

        Logs information about calculation steps, computes the average
        pixel size based on image or cell scaling settings,
        and sets initial thresholds for object detection.

        Notes
        -----
        - The average pixel size is determined by either image dimensions or blob sizes.
        - Thresholds for automatic detection are set based on configuration settings.

        """
        logging.info("Getting average pixel size")
        (self.first_image.shape_number,
            self.first_image.shapes,
            self.first_image.stats,
            centroids) = cv2.connectedComponentsWithStats(
                self.first_image.validated_shapes,
                connectivity=8)
        self.first_image.shape_number -= 1
        self.vars['average_pixel_size']: float = 1.
        if self.all['scale_with_image_or_cells'] == 0:
            self.vars['average_pixel_size'] = float(np.square(self.all['image_horizontal_size_in_mm'] /
                                                        self.first_im.shape[1]))
        else:
            if len(self.first_image.stats[1:, 2]) > 0:
                self.vars['average_pixel_size'] = float(np.square(self.all['starting_blob_hsize_in_mm'] /
                                                            np.mean(self.first_image.stats[1:, 2])))
            else:
                self.vars['output_in_mm'] = False

        if self.all['set_spot_size']:
            self.starting_blob_hsize_in_pixels = float((self.all['starting_blob_hsize_in_mm'] /
                                                  np.sqrt(self.vars['average_pixel_size'])))
        else:
            self.starting_blob_hsize_in_pixels = None

        if not self.all['automatic_size_thresholding']:
            self.vars['first_move_threshold'] = int(np.round(self.all['first_move_threshold_in_mm²'] /
                                                         self.vars['average_pixel_size']))
        logging.info(f"The average pixel size is: {self.vars['average_pixel_size']} mm²")

    def get_background_to_subtract(self):
        """
        Determine if background subtraction should be applied to the image.

        Extended Description
        --------------------
        This function checks whether background subtraction should be applied.
        It utilizes the 'subtract_background' flag and potentially converts
        the image for motion estimation.

        Parameters
        ----------
        self : object
            The instance of the class containing this method.
            Must have attributes `vars` and `first_image`.
        """
        if self.vars['subtract_background']:
            self.first_image.generate_subtract_background(self.vars['convert_for_motion'], self.vars['drift_already_corrected'])

    def find_if_lighter_background(self):
        """
        Determines whether the background is lighter or darker than the cells.

        This function analyzes images to determine if their backgrounds are lighter
        or darker relative to the cells, updating attributes accordingly for analysis and display purposes.


        Notes
        -----
        This function modifies instance variables and does not return any value.
        The analysis involves comparing mean pixel values in specific areas of the image.
        """
        logging.info("Find if the background is lighter or darker than the cells")
        self.vars['lighter_background']: bool = True
        self.vars['contour_color']: np.uint8 = 0
        are_dicts_equal: bool = True
        if self.vars['convert_for_origin'] is not None and self.vars['convert_for_origin'] is not None:
            for key in self.vars['convert_for_origin'].keys():
                are_dicts_equal = are_dicts_equal and np.all(key in self.vars['convert_for_motion'] and self.vars['convert_for_origin'][key] == self.vars['convert_for_motion'][key])

            for key in self.vars['convert_for_motion'].keys():
                are_dicts_equal = are_dicts_equal and np.all(key in self.vars['convert_for_origin'] and self.vars['convert_for_motion'][key] == self.vars['convert_for_origin'][key])
        else:
            self.vars['convert_for_origin'] = {"logical": 'None', "PCA": [1, 1, 1]}
        if are_dicts_equal:
            if self.first_im is None:
                self.get_first_image()
                self.fast_first_image_segmentation()
                self.cropping(is_first_image=True)
            among = np.nonzero(self.first_image.validated_shapes)
            not_among = np.nonzero(1 - self.first_image.validated_shapes)
            # Use the converted image to tell if the background is lighter, for analysis purposes
            if self.first_image.image[among[0], among[1]].mean() > self.first_image.image[not_among[0], not_among[1]].mean():
                self.vars['lighter_background'] = False
            # Use the original image to tell if the background is lighter, for display purposes
            if self.first_image.bgr[among[0], among[1], ...].mean() > self.first_image.bgr[not_among[0], not_among[1], ...].mean():
                self.vars['contour_color'] = 255
        else:
            if self.last_im is None:
                self.get_last_image()
                # self.cropping(is_first_image=False)
                self.fast_last_image_segmentation()
            if self.last_image.binary_image.sum() == 0:
                self.fast_last_image_segmentation()
            among = np.nonzero(self.last_image.binary_image)
            not_among = np.nonzero(1 - self.last_image.binary_image)
            # Use the converted image to tell if the background is lighter, for analysis purposes
            if self.last_image.image[among[0], among[1]].mean() > self.last_image.image[not_among[0], not_among[1]].mean():
                self.vars['lighter_background'] = False
            # Use the original image to tell if the background is lighter, for display purposes
            if self.last_image.bgr[among[0], among[1], ...].mean() > self.last_image.bgr[not_among[0], not_among[1], ...].mean():
                self.vars['contour_color'] = 255
        if self.vars['origin_state'] == "invisible":
            binary_image = self.first_image.binary_image.copy()
            self.first_image.convert_and_segment(self.vars['convert_for_motion'], self.vars["color_number"],
                                                 None, None, subtract_background=None,
                                                 subtract_background2=None,
                                                 rolling_window_segmentation=self.vars['rolling_window_segmentation'],
                                                 filter_spec=self.vars["filter_spec"])
            covered_values = self.first_image.image[np.nonzero(binary_image)]
            self.vars['luminosity_threshold'] = 127
            if len(covered_values) > 0:
                if self.vars['lighter_background']:
                    if np.max(covered_values) < 255:
                        self.vars['luminosity_threshold'] = np.max(covered_values) + 1
                else:
                    if np.min(covered_values) > 0:
                        self.vars['luminosity_threshold'] = np.min(covered_values) - 1

    def delineate_each_arena(self):
        """
        Determine the coordinates of each arena for video analysis.

        The function processes video frames to identify bounding boxes around
        specimens and determines valid arenas for analysis. In case of existing data,
        it uses previously computed coordinates if available and valid.

        Returns
        -------
        analysis_status : dict
            A dictionary containing flags and messages indicating the status of
            the analysis.
            - 'continue' (bool): Whether to continue processing.
            - 'message' (str): Informational or error message.

        Notes
        -----
        This function relies on the existence of certain attributes and variables
        defined in the class instance.
        """
        analysis_status = {"continue": True, "message": ""}
        if not self.vars['several_blob_per_arena'] and (self.sample_number > 1):
            motion_list = None
            if self.all['are_gravity_centers_moving']:
                motion_list = self._segment_blob_motion(sample_size=5)
            self.get_bounding_boxes(are_gravity_centers_moving=self.all['are_gravity_centers_moving'] == 1,
                motion_list=motion_list, all_specimens_have_same_direction=self.all['all_specimens_have_same_direction'])

            if np.any(self.ordered_stats[:, 4] > 100 * np.median(self.ordered_stats[:, 4])):
                analysis_status['message'] = "A specimen is at least 100 times larger: click previous and retry by specifying 'back' areas."
                analysis_status['continue'] = False
            if np.any(self.ordered_stats[:, 4] < 0.01 * np.median(self.ordered_stats[:, 4])):
                analysis_status['message'] = "A specimen is at least 100 times smaller: click previous and retry by specifying 'back' areas."
                analysis_status['continue'] = False
            del self.ordered_stats
            logging.info(
                str(self.not_analyzed_individuals) + " individuals are out of picture scope and cannot be analyzed")
        else:
            self._whole_image_bounding_boxes()
            self.sample_number = 1
        self._set_analyzed_individuals()
        self.vars['arena_coord'] = []
        self.save_coordinates()
        return analysis_status

    def _segment_blob_motion(self, sample_size: int) -> list:
        """
        Segment blob motion from the data list at specified sample sizes.

        Parameters
        ----------
        sample_size : int
            Number of samples to take from the data list.

        Returns
        -------
        list
            List containing segmented binary images at sampled frames.

        Notes
        -----
        This function uses numpy for handling array operations and assumes the presence of certain attributes in the object, namely `data_list`, `first_image`, and `vars`.

        Examples
        --------
        >>> motion_samples = _segment_blob_motion(10)
        >>> print(len(motion_samples))  # Expected output: 10
        """
        motion_list = list()
        if isinstance(self.data_list, list):
            frame_number = len(self.data_list)
        else:
            frame_number = self.data_list.shape[0]
        sample_numbers = np.floor(np.linspace(0, frame_number, sample_size)).astype(int)
        if not 'lighter_background' in self.vars.keys():
            self.find_if_lighter_background()
        for frame_idx in np.arange(sample_size):
            if frame_idx == 0:
                motion_list.insert(frame_idx, self.first_image.validated_shapes)
            else:
                if isinstance(self.data_list[0], str):
                    image = self.data_list[sample_numbers[frame_idx] - 1]
                else:
                    image = self.data_list[sample_numbers[frame_idx] - 1]
                if isinstance(image, str):
                    is_landscape = self.first_image.image.shape[0] < self.first_image.image.shape[1]
                    image = read_and_rotate(image, self.first_image.bgr, self.all['raw_images'],
                                            is_landscape, self.first_image.crop_coord)
                    # image = readim(image)
                In = OneImageAnalysis(image)
                if self.vars['drift_already_corrected']:
                    In.check_if_image_border_attest_drift_correction()
                    # In.adjust_to_drift_correction(self.vars['convert_for_motion']['logical'])
                In.convert_and_segment(self.vars['convert_for_motion'], self.vars['color_number'], None, None,
                                       self.first_image.subtract_background, self.first_image.subtract_background2,
                                       self.vars['rolling_window_segmentation'], self.vars['lighter_background'],
                                       allowed_window=In.drift_mask_coord, filter_spec=self.vars['filter_spec'])
                motion_list.insert(frame_idx, In.binary_image)
        return motion_list


    def get_bounding_boxes(self, are_gravity_centers_moving: bool, motion_list: list=(), all_specimens_have_same_direction: bool=True, original_shape_hsize: int=None):
        """Get the coordinates of arenas using bounding boxes.

        Parameters
        ----------
        are_gravity_centers_moving : bool
            Flag indicating whether gravity centers are moving or not.
        motion_list : list
            List of motion information for the specimens.
        all_specimens_have_same_direction : bool, optional
            Flag indicating whether all specimens have the same direction,
            by default True.
        Notes
        -----
        This method uses various internal methods and variables to determine the bounding boxes.
        """
        # 7) Create required empty arrays: especially the bounding box coordinates of each video
        self.ordered_first_image = None
        self.shapes_to_remove = None
        if self.first_image.y_boundaries is None:
            self.first_image.get_setup_boundaries()

        logging.info("Get the coordinates of all arenas using the get_bounding_boxes method")
        if self.first_image.validated_shapes.any() and self.first_image.shape_number > 0:
            self.ordered_stats, ordered_centroids, self.ordered_first_image = rank_from_top_to_bottom_from_left_to_right(
                self.first_image.validated_shapes, self.first_image.y_boundaries, get_ordered_image=True)
            self.unchanged_ordered_fimg = self.ordered_first_image.copy()
            self.modif_validated_shapes = self.first_image.validated_shapes.copy()
            self.standard = - 1
            counter = 0
            while np.any(np.less(self.standard, 0)) and counter < 20:
                counter += 1
                self.left = np.zeros(self.first_image.shape_number, dtype=np.int64)
                self.right = np.repeat(self.modif_validated_shapes.shape[1], self.first_image.shape_number)
                self.top = np.zeros(self.first_image.shape_number, dtype=np.int64)
                self.bot = np.repeat(self.modif_validated_shapes.shape[0], self.first_image.shape_number)
                if are_gravity_centers_moving:
                    self.top, self.bot, self.left, self.right, self.ordered_first_image = get_bb_with_moving_centers(motion_list, all_specimens_have_same_direction,
                                                     original_shape_hsize, self.first_image.validated_shapes,
                                                     self.first_image.y_boundaries)
                    new_ordered_first_image = np.zeros(self.ordered_first_image.shape, dtype=np.uint8)

                    for i in np.arange(1, self.first_image.shape_number + 1):
                        previous_shape = np.zeros(self.ordered_first_image.shape, dtype=np.uint8)
                        previous_shape[np.nonzero(self.unchanged_ordered_fimg == i)] = 1
                        new_potentials = np.zeros(self.ordered_first_image.shape, dtype=np.uint8)
                        new_potentials[np.nonzero(self.ordered_first_image == i)] = 1
                        new_potentials[np.nonzero(self.unchanged_ordered_fimg == i)] = 0

                        pads = ProgressivelyAddDistantShapes(new_potentials, previous_shape, max_distance=2)
                        pads.consider_shapes_sizes(min_shape_size=10)
                        pads.connect_shapes(only_keep_connected_shapes=True, rank_connecting_pixels=False)
                        new_ordered_first_image[np.nonzero(pads.expanded_shape)] = i
                    self.ordered_first_image = new_ordered_first_image
                    self.modif_validated_shapes = np.zeros(self.ordered_first_image.shape, dtype=np.uint8)
                    self.modif_validated_shapes[np.nonzero(self.ordered_first_image)] = 1
                    self.ordered_stats, ordered_centroids, self.ordered_first_image = rank_from_top_to_bottom_from_left_to_right(
                        self.modif_validated_shapes, self.first_image.y_boundaries, get_ordered_image=True)
                    self.top, self.bot, self.left, self.right = get_quick_bounding_boxes(self.modif_validated_shapes, self.ordered_first_image, self.ordered_stats)
                else:
                    self.top, self.bot, self.left, self.right = get_quick_bounding_boxes(self.modif_validated_shapes, self.ordered_first_image, self.ordered_stats)
                self._standardize_video_sizes()
            if counter == 20:
                self.top[self.top < 0] = 1
                self.bot[self.bot >= self.ordered_first_image.shape[0] - 1] = self.ordered_first_image.shape[0] - 1
                self.left[self.left < 0] = 1
                self.right[self.right >= self.ordered_first_image.shape[1] - 1] = self.ordered_first_image.shape[1] - 1
            del self.ordered_first_image
            del self.unchanged_ordered_fimg
            del self.modif_validated_shapes
            del self.standard
            del self.shapes_to_remove
            self.bot += 1
            self.right += 1
        else:
            self._whole_image_bounding_boxes()

    def _whole_image_bounding_boxes(self):
        self.top, self.bot, self.left, self.right = np.array([0]), np.array([self.first_image.image.shape[0]]), np.array([0]), np.array([self.first_image.image.shape[1]])

    def _standardize_video_sizes(self):
        """
        Standardize video sizes by adjusting bounding boxes.

        Extended Description
        --------------------
        This function adjusts the bounding boxes of detected shapes in a video frame.
        It ensures that all bounding boxes are within the frame's boundaries and
        standardizes their sizes to avoid issues with odd dimensions during video writing.

        Returns
        -------
        None
            The function modifies the following attributes of the class instance:

        Attributes Modified
        ------------------
        standard : numpy.ndarray
            Standardized bounding boxes.
        shapes_to_remove : numpy.ndarray
            Indices of shapes to be removed from the image.
        modif_validated_shapes : numpy.ndarray
            Modified validated shapes after removing out-of-picture areas.
        ordered_stats : list of float
            Updated order statistics for the shapes.
        ordered_centroids : numpy.ndarray
            Centroids of the ordered shapes.
        ordered_first_image : numpy.ndarray
            First image with updated order statistics and centroids.
        first_image.shape_number : int
            Updated number of shapes in the first image.
        not_analyzed_individuals : numpy.ndarray
            Indices of individuals not analyzed after modifications.

        """
        distance_threshold_to_consider_an_arena_out_of_the_picture = None# in pixels, worked nicely with - 50

        # The modifications allowing to not make videos of setups out of view, do not work for moving centers
        y_diffs = self.bot - self.top
        x_diffs = self.right - self.left
        add_to_y = ((np.max(y_diffs) - y_diffs) / 2)
        add_to_x = ((np.max(x_diffs) - x_diffs) / 2)
        self.standard = np.zeros((len(self.top), 4), dtype=np.int64)
        self.standard[:, 0] = self.top - np.uint8(np.floor(add_to_y))
        self.standard[:, 1] = self.bot + np.uint8(np.ceil(add_to_y))
        self.standard[:, 2] = self.left - np.uint8(np.floor(add_to_x))
        self.standard[:, 3] = self.right + np.uint8(np.ceil(add_to_x))

        # Monitor if one bounding box gets out of picture shape
        out_of_pic = self.standard.copy()
        out_of_pic[:, 1] = self.ordered_first_image.shape[0] - out_of_pic[:, 1] - 1
        out_of_pic[:, 3] = self.ordered_first_image.shape[1] - out_of_pic[:, 3] - 1

        if distance_threshold_to_consider_an_arena_out_of_the_picture is None:
            distance_threshold_to_consider_an_arena_out_of_the_picture = np.min(out_of_pic) - 1

        # If it occurs at least one time, apply a correction, otherwise, continue and write videos
        # If the overflow is strong, remove the corresponding individuals and remake bounding_box finding
        if np.any(np.less(out_of_pic, distance_threshold_to_consider_an_arena_out_of_the_picture)):
            # Remove shapes
            self.standard = - 1
            self.shapes_to_remove = np.nonzero(np.less(out_of_pic, - 20))[0]
            for shape_i in self.shapes_to_remove:
                self.ordered_first_image[self.ordered_first_image == (shape_i + 1)] = 0
            self.modif_validated_shapes = np.zeros(self.ordered_first_image.shape, dtype=np.uint8)
            self.modif_validated_shapes[np.nonzero(self.ordered_first_image)] = 1
            self.ordered_stats, ordered_centroids, self.ordered_first_image = rank_from_top_to_bottom_from_left_to_right(
                self.modif_validated_shapes, self.first_image.y_boundaries, get_ordered_image=True)

            self.first_image.shape_number = self.first_image.shape_number - len(self.shapes_to_remove)
            self.not_analyzed_individuals = np.unique(self.unchanged_ordered_fimg -
                                                      (self.unchanged_ordered_fimg * self.modif_validated_shapes))[1:]

        else:
            # Reduce all box sizes if necessary and proceed
            if np.any(np.less(out_of_pic, 0)):
                # When the overflow is weak, remake standardization with lower "add_to_y" and "add_to_x"
                overflow = np.nonzero(np.logical_and(np.less(out_of_pic, 0), np.greater_equal(out_of_pic, distance_threshold_to_consider_an_arena_out_of_the_picture)))[0]
                # Look if overflow occurs on the y axis
                if np.any(np.less(out_of_pic[overflow, :2], 0)):
                    add_to_top_and_bot = np.min(out_of_pic[overflow, :2])
                    self.standard[:, 0] = self.standard[:, 0] - add_to_top_and_bot
                    self.standard[:, 1] = self.standard[:, 1] + add_to_top_and_bot
                # Look if overflow occurs on the x axis
                if np.any(np.less(out_of_pic[overflow, 2:], 0)):
                    add_to_left_and_right = np.min(out_of_pic[overflow, 2:])
                    self.standard[:, 2] = self.standard[:, 2] - add_to_left_and_right
                    self.standard[:, 3] = self.standard[:, 3] + add_to_left_and_right
            # If x or y sizes are odd, make them even :
            # Don't know why, but opencv remove 1 to odd shapes when writing videos
            if (self.standard[0, 1] - self.standard[0, 0]) % 2 != 0:
                self.standard[:, 1] -= 1
            if (self.standard[0, 3] - self.standard[0, 2]) % 2 != 0:
                self.standard[:, 3] -= 1
            self.top = self.standard[:, 0]
            self.bot = self.standard[:, 1]
            self.left = self.standard[:, 2]
            self.right = self.standard[:, 3]

    def save_origins_and_backgrounds_lists(self):
        """
        Create origins and background lists for image processing.

        Extended Description
        --------------------
        This method generates the origin and background lists by slicing the first image
        and its background subtraction based on predefined boundaries. It handles cases where
        the top, bottom, left, and right boundaries are not yet initialized.

        Notes
        -----
        This method directly modifies the input image data. The `self.vars` dictionary is populated
        with lists of sliced arrays from the first image and its background.

        Attributes
        ----------
        self.vars : dict
            Dictionary to store processed data.
        self.first_image : ImageObject
            The first image object containing validated shapes and background subtraction arrays.
        """
        logging.info("Create origins and background lists")
        if self.top is None:
            self._whole_image_bounding_boxes()

        if not self.first_image.validated_shapes.any():
            if self.vars['convert_for_motion'] is not None:
                self.vars['convert_for_origin'] = self.vars['convert_for_motion']
            self.fast_first_image_segmentation()
        first_im = self.first_image.validated_shapes
        for rep, arena_label in enumerate(self.vars['analyzed_individuals']):
            origin_coord = np.nonzero(first_im[self.top[rep]:self.bot[rep], self.left[rep]:self.right[rep]])
            write_h5(f'ind_{arena_label}.h5', np.array(origin_coord), 'origin_coord')
        if self.vars['subtract_background']:
            if self.first_image.subtract_background is None:
                self.get_background_to_subtract()
            for rep in np.arange(len(self.vars['analyzed_individuals'])):
                background = self.first_image.subtract_background[self.top[rep]:self.bot[rep], self.left[rep]:self.right[rep]]
                write_h5(f'ind_{arena_label}.h5', background, 'background')
                if self.vars['convert_for_motion']['logical'] != 'None':
                    background2 = self.first_image.subtract_background2[self.top[rep]:self.bot[rep], self.left[rep]:self.right[rep]]
                    write_h5(f'ind_{arena_label}.h5', background2, 'background2')

    def complete_image_analysis(self):
        if not self.visualize and len(self.last_image.im_combinations) > 0:
            self.last_image.binary_image = self.last_image.im_combinations[self.current_combination_id]['binary_image']
            self.last_image.image = self.last_image.im_combinations[self.current_combination_id]['converted_image']
        self.instantiate_tables()
        if len(self.vars['exif']) > 1:
            self.vars['exif'] = self.vars['exif'][0]
        if len(self.last_image.all_c_spaces) == 0:
            self.last_image.all_c_spaces['bgr'] = self.last_image.bgr.copy()
        if self.bio_mask is not None:
            self.last_image.binary_image[self.bio_mask] = 1
        if self.back_mask is not None:
            self.last_image.binary_image[self.back_mask] = 0
        for i, arena in enumerate(self.vars['analyzed_individuals']):
            binary = self.last_image.binary_image[self.top[i]:self.bot[i], self.left[i]:self.right[i]]
            efficiency_test = self.last_image.all_c_spaces['bgr'][self.top[i]:self.bot[i], self.left[i]:self.right[i], :]
            if not self.vars['several_blob_per_arena']:
                binary = keep_one_connected_component(binary)
                one_row_per_frame = compute_one_descriptor_per_frame(binary[None, :, :],
                                                                     arena,
                                                                     self.vars['exif'],
                                                                     self.vars['descriptors'],
                                                                     self.vars['output_in_mm'],
                                                                     self.vars['average_pixel_size'],
                                                                     self.vars['specimen_activity'],
                                                                     self.vars['save_coord_specimen'])
                coord_network = None
                coord_pseudopods = None
                if self.vars['save_graph']:
                    if coord_network is None:
                        coord_network = np.array(np.nonzero(binary))
                    extract_graph_dynamics(self.last_image.image[None, :, :], coord_network, arena,
                                           0, None, coord_pseudopods)

            else:
                one_row_per_frame = compute_one_descriptor_per_colony(binary[None, :, :],
                                                                      arena,
                                                                      self.vars['exif'],
                                                                      self.vars['descriptors'],
                                                                      self.vars['output_in_mm'],
                                                                      self.vars['average_pixel_size'],
                                                                      self.vars['specimen_activity'],
                                                                      self.vars['first_move_threshold'],
                                                                      self.vars['save_coord_specimen'])
            if self.vars['fractal_analysis']:
                zoomed_binary, side_lengths = prepare_box_counting(binary,
                                                                   min_mesh_side=self.vars[
                                                                       'fractal_box_side_threshold'],
                                                                   zoom_step=self.vars['fractal_zoom_step'],
                                                                   contours=True)
                box_counting_dimensions = box_counting_dimension(zoomed_binary, side_lengths)
                one_row_per_frame["fractal_dimension"] = box_counting_dimensions[0]
                one_row_per_frame["fractal_box_nb"] = box_counting_dimensions[1]
                one_row_per_frame["fractal_r_value"] = box_counting_dimensions[2]

            one_descriptor_per_arena = {}
            one_descriptor_per_arena["arena"] = arena
            one_descriptor_per_arena["first_move"] = pd.NA
            one_descriptor_per_arena["final_area"] = binary.sum()
            one_descriptor_per_arena["iso_digi_transi"] = pd.NA
            one_descriptor_per_arena["is_growth_isotropic"] = pd.NA
            self.update_one_row_per_arena(i, one_descriptor_per_arena)
            self.update_one_row_per_frame(i * 1, (i + 1) * 1, one_row_per_frame)
            contours = np.nonzero(get_contours(binary))
            efficiency_test[contours[0], contours[1], :] = np.array((94, 0, 213), dtype=np.uint8)
            self.add_analysis_visualization_to_first_and_last_images(i, efficiency_test, None)
        self.save_tables(with_last_image=False)

    def prepare_video_writing(self, img_list: list, min_ram_free: float, in_colors: bool=False, pathway: str=""):
        """

        Prepare the raw video (.h5) writing process for Cellects.

        Parameters
        ----------
        img_list : list
            List of images to be processed.
        min_ram_free : float
            Minimum amount of RAM in GB that should remain free.
        in_colors : bool, optional
            Whether the images are in color. Default is False.
        pathway : str, optional
            Path to save the video files. Default is an empty string.

        Returns
        -------
        tuple
            A tuple containing:
            - bunch_nb: int, number of bunches needed for video writing.
            - video_nb_per_bunch: int, number of videos per bunch.
            - sizes: ndarray, dimensions of each video.
            - video_bunch: list or ndarray, initialized video arrays.
            - vid_names: list, names of the video files.
            - rom_memory_required: None or float, required ROM memory.
            - analysis_status: dict, status and message of the analysis process.
            - remaining: int, remainder videos that do not fit in a complete bunch.

        Notes
        -----
        - The function calculates necessary memory and ensures 10% extra to avoid issues.
        - It checks for available RAM and adjusts the number of bunches accordingly.
        - If using color images, memory requirements are tripled.

        expected output depends on the provided images and RAM availability
        """
        # 1) Create a list of video names
        if self.not_analyzed_individuals is not None:
            number_to_add = len(self.not_analyzed_individuals)
        else:
            number_to_add = 0
        vid_names = list()
        ind_i = 0
        counter = 0
        while ind_i < (self.first_image.shape_number + number_to_add):
            ind_i += 1
            while np.any(np.isin(self.not_analyzed_individuals, ind_i)):
                ind_i += 1
            vid_names.append(pathway + "ind_" + str(ind_i) + ".h5")
            counter += 1
        img_nb = len(img_list)

        # 2) Create a table of the dimensions of each video
        # Add 10% to the necessary memory to avoid problems
        vertical_diffs = np.array(self.bot) - np.array(self.top)
        horizontal_diffs = np.array(self.right) - np.array(self.left)
        necessary_memory = img_nb * np.multiply((vertical_diffs).astype(np.uint64), (horizontal_diffs).astype(np.uint64)).sum() * 8 * 1.16415e-10
        if in_colors:
            sizes = np.column_stack(
                (np.repeat(img_nb, self.first_image.shape_number), vertical_diffs, horizontal_diffs,
                 np.repeat(3, self.first_image.shape_number)))
            necessary_memory *= 3
        else:
            sizes = np.column_stack(
                (np.repeat(img_nb, self.first_image.shape_number), vertical_diffs, horizontal_diffs))
        use_list_of_vid = True
        if np.all(sizes[0, :] == sizes):
            use_list_of_vid = False
        available_memory = (psutil.virtual_memory().available >> 30) - min_ram_free
        if available_memory == 0:
            analysis_status = {"continue": False, "message": "There are not enough RAM available"}
            bunch_nb = 1
        else:
            bunch_nb = int(np.ceil(necessary_memory / available_memory))
            if bunch_nb > 1:
                # The program will need twice the memory to create the second bunch.
                bunch_nb = int(np.ceil(2 * necessary_memory / available_memory))

        video_nb_per_bunch = np.floor(self.first_image.shape_number / bunch_nb).astype(np.uint8)
        analysis_status = {"continue": True, "message": ""}
        video_bunch = None
        try:
            if use_list_of_vid:
                video_bunch = [np.zeros(sizes[i, :], dtype=np.uint8) for i in range(video_nb_per_bunch)]
            else:
                video_bunch = np.zeros(np.append(sizes[0, :], video_nb_per_bunch), dtype=np.uint8)
        except ValueError as v_err:
            analysis_status = {"continue": False, "message": "Probably failed to detect the right cell(s) number, do the first image analysis manually."}
            logging.error(f"{analysis_status['message']} error is: {v_err}")
        # Check for available ROM memory
        if (psutil.disk_usage('/')[2] >> 30) < (necessary_memory + 2):
            rom_memory_required = necessary_memory + 2
        else:
            rom_memory_required = None
        remaining = self.first_image.shape_number % bunch_nb
        if remaining > 0:
            bunch_nb += 1
        is_landscape = self.first_image.image.shape[0] < self.first_image.image.shape[1]
        logging.info(f"Cellects will start writing {self.first_image.shape_number} videos. Given available memory, it will do it in {bunch_nb} time(s)")
        return bunch_nb, video_nb_per_bunch, sizes, video_bunch, vid_names, rom_memory_required, analysis_status, remaining, use_list_of_vid, is_landscape


    def update_output_list(self):
        """
        Update the output list with various descriptors from the analysis results.

        This method processes different types of descriptors and assigns them to
        the `self.vars['descriptors']` dictionary. It handles special cases for
        descriptors related to 'xy' dimensions and ensures that all relevant metrics
        are stored in the output list.
        """
        self.vars['descriptors'] = {}
        for descriptor in self.all['descriptors'].keys():
            if descriptor == 'standard_deviation_xy':
                self.vars['descriptors']['standard_deviation_x'] = self.all['descriptors'][descriptor]
                self.vars['descriptors']['standard_deviation_y'] = self.all['descriptors'][descriptor]
            elif descriptor == 'skewness_xy':
                self.vars['descriptors']['skewness_x'] = self.all['descriptors'][descriptor]
                self.vars['descriptors']['skewness_y'] = self.all['descriptors'][descriptor]
            elif descriptor == 'kurtosis_xy':
                self.vars['descriptors']['kurtosis_x'] = self.all['descriptors'][descriptor]
                self.vars['descriptors']['kurtosis_y'] = self.all['descriptors'][descriptor]
            elif descriptor == 'major_axes_len_and_angle':
                self.vars['descriptors']['major_axis_len'] = self.all['descriptors'][descriptor]
                self.vars['descriptors']['minor_axis_len'] = self.all['descriptors'][descriptor]
                self.vars['descriptors']['axes_orientation'] = self.all['descriptors'][descriptor]
            else:
                if np.isin(descriptor, list(from_shape_descriptors_class.keys())):

                    self.vars['descriptors'][descriptor] = self.all['descriptors'][descriptor]
        self.vars['descriptors']['newly_explored_area'] = self.vars['specimen_activity'] == 'move' or self.vars['specimen_activity'] == 'move and grow'

    def update_available_core_nb(self, image_bit_number=256, video_bit_number=140):# video_bit_number=176
        """
        Update available computation resources based on memory and processing constraints.

        Parameters
        ----------
        image_bit_number : int, optional
            Number of bits per image pixel (default is 256).
        video_bit_number : int, optional
            Number of bits per video frame pixel (default is 140).

        Other Parameters
        ----------------
        lose_accuracy_to_save_memory : bool
            Flag to reduce accuracy for memory savings.
        convert_for_motion : dict
            Conversion settings for motion analysis.
        already_greyscale : bool
            Flag indicating if the image is already greyscale.
        save_coord_thickening_slimming : bool
            Flag to save coordinates for thickening and slimming.
        oscilacyto_analysis : bool
            Flag indicating if oscilacyto analysis is enabled.
        save_coord_network : bool
            Flag to save coordinates for network analysis.

        Returns
        -------
        float
            Rounded absolute difference between available memory and necessary memory in GB.

        Notes
        -----
        Performance considerations and limitations should be noted here if applicable.

        """
        if self.vars['lose_accuracy_to_save_memory']:
            video_bit_number -= 56
        if self.vars['convert_for_motion']['logical'] != 'None':
            video_bit_number += 64
            if self.vars['lose_accuracy_to_save_memory']:
                video_bit_number -= 56
        if self.vars['already_greyscale']:
            video_bit_number -= 64
        if self.vars['save_coord_thickening_slimming'] or self.vars['oscilacyto_analysis']:
            video_bit_number += 16
            image_bit_number += 128
        if self.vars['save_coord_network']:
            video_bit_number += 8
            image_bit_number += 64

        if isinstance(self.bot, list):
            one_image_memory = np.multiply((self.bot[0] - self.top[0]),
                                        (self.right[0] - self.left[0])).max().astype(np.uint64)
        else:
            one_image_memory = np.multiply((self.bot - self.top).astype(np.uint64),
                                        (self.right - self.left).astype(np.uint64)).max()
        one_video_memory = self.vars['img_number'] * one_image_memory
        necessary_memory = (one_image_memory * image_bit_number + one_video_memory * video_bit_number) * 1.16415e-10
        available_memory = (virtual_memory().available >> 30) - self.vars['min_ram_free']
        max_repeat_in_memory = int(available_memory // necessary_memory)
        if max_repeat_in_memory > 1:
            max_repeat_in_memory = max(int(available_memory // (2 * necessary_memory)), 1)


        self.cores = min(self.all['cores'], max_repeat_in_memory)
        if self.cores > self.sample_number:
            self.cores = self.sample_number
        return np.round(np.absolute(available_memory - necessary_memory), 3)


    def update_one_row_per_arena(self, i: int, table_to_add):
        """
        Update one row of the dataframe per arena.

        Add a row to a DataFrame for each arena, based on the provided table_to_add. If no previous rows exist,
        initialize a new DataFrame with zeros.

        Parameters
        ----------
        i : int
            Index of the arena to update.
        table_to_add : dict
            Dictionary containing values to add. Keys are column names, values are the data.

        """
        if not self.vars['several_blob_per_arena']:
            if self.one_row_per_arena is None:
                self.one_row_per_arena = pd.DataFrame(np.zeros((len(self.vars['analyzed_individuals']), len(table_to_add)), dtype=float),
                                            columns=table_to_add.keys())
            self.one_row_per_arena.iloc[i, :] = table_to_add.values()


    def update_one_row_per_frame(self, i: int, j: int, table_to_add):
        """
        Update a range of rows in `self.one_row_per_frame` DataFrame with values from
        `table_to_add`.

        Parameters
        ----------
        i : int
            The starting row index to update in `self.one_row_per_frame`.
        j : int
            The ending row index (exclusive) to update in `self.one_row_per_frame`.
        table_to_add : dict
            A dictionary where keys are column labels and values are lists or arrays of
            data to insert into `self.one_row_per_frame`.
        Notes
        -----
        Ensures that one row per arena is being updated. If `self.one_row_per_frame` is
        None, it initializes a DataFrame to hold the data.
        """
        if not self.vars['several_blob_per_arena']:
            if self.one_row_per_frame is None:
                self.one_row_per_frame = pd.DataFrame(index=range(len(self.vars['analyzed_individuals']) *
                                                        self.vars['img_number']),
                                            columns=table_to_add.keys())

            self.one_row_per_frame.iloc[i:j, :] = table_to_add


    def instantiate_tables(self):
        """
        Update output list and prepare results tables and validation images.

        Extended Description
        --------------------
        This method performs necessary preparations for processing image sequences,
        including updating the output list and initializing key attributes required
        for subsequent operations.

        """
        self.update_output_list()
        logging.info("Instantiate results tables and validation images")
        self.fractal_box_sizes = None
        self.one_row_per_arena = None
        self.one_row_per_frame = None
        if self.vars['already_greyscale']:
            if len(self.first_image.bgr.shape) == 2:
                self.first_image.bgr = np.stack((self.first_image.bgr, self.first_image.bgr, self.first_image.bgr), axis=2).astype(np.uint8)
            if len(self.last_image.bgr.shape) == 2:
                self.last_image.bgr = np.stack((self.last_image.bgr, self.last_image.bgr, self.last_image.bgr), axis=2).astype(np.uint8)
            self.vars["convert_for_motion"] = {"bgr": [1, 1, 1], "logical": "None"}

    def add_analysis_visualization_to_first_and_last_images(self, i: int, first_visualization: NDArray, last_visualization: NDArray=None):
        """
        Adds analysis visualizations to the first and last images of a sequence.

        Parameters
        ----------
        i : int
            Index of the image in the sequence.
        first_visualization : NDArray[np.uint8]
            The visualization to add to the first image.
        last_visualization : NDArray[np.uint8]
            The visualization to add to the last image.

        Other Parameters
        ----------------
        vars : dict
            Dictionary containing various parameters.
        arena_shape : str, optional
            The shape of the arena. Either 'circle' or other shapes.

        Notes
        -----
        If `arena_shape` is 'circle', the visualization will be masked by an ellipse.

        """
        minmax = (self.top[i], self.bot[i], self.left[i], self.right[i])
        self.first_image.bgr = draw_img_with_mask(self.first_image.bgr, self.first_image.bgr.shape[:2], minmax,
                                                  self.vars['arena_shape'], first_visualization)
        if last_visualization is not None:
            self.last_image.bgr = draw_img_with_mask(self.last_image.bgr, self.last_image.bgr.shape[:2], minmax,
                                                      self.vars['arena_shape'], last_visualization)


    def save_tables(self, with_last_image: bool=True):
        """
        Exports analysis results to CSV files and saves visualization outputs.

        Generates the following output:
        - one_row_per_arena.csv, one_row_per_frame.csv : Tracking data per arena/frame.
        - cellects_settings.json : Full configuration settings for reproducibility.

        Parameters
        ----------
        with_last_image : bool, optional
            Also save the last image. Create duplicate when the analysis contains only one image.

        Raises
        ------
        PermissionError
            If any output file is already open in an external program (logged and re-raised).

        Notes
        -----
        Ensure no exported CSV files are open while running this method to avoid permission errors. This
        function will fail gracefully if the files cannot be overwritten.

        """
        logging.info("Save results tables and validation images")
        if not self.vars['several_blob_per_arena']:
            try:
                self.one_row_per_arena.to_csv("one_row_per_arena.csv", sep=";", index=False, lineterminator='\n')
                del self.one_row_per_arena
            except PermissionError:
                logging.error("Never let one_row_per_arena.csv open when Cellects runs")
            try:
                self.one_row_per_frame.to_csv("one_row_per_frame.csv", sep=";", index=False, lineterminator='\n')
                del self.one_row_per_frame
            except PermissionError:
                logging.error("Never let one_row_per_frame.csv open when Cellects runs")
        if self.all['extension'] == '.JPG':
            extension = '.PNG'
        else:
            extension = '.JPG'
        if with_last_image:
            cv2.imwrite(f"Analysis efficiency, last image{extension}", self.last_image.bgr)
        cv2.imwrite(
            f"Analysis efficiency, {np.ceil(self.vars['img_number'] / 10).astype(np.uint64)}th image{extension}",
            self.first_image.bgr)

__init__()

This class stores all variables required for analysis as well as methods to process it. Global variables (i.e. that does not concern the MotionAnalysis) are directly stored in self. Variables used in the MotionAnalysis class are stored in a dict called self.vars

Source code in src/cellects/core/program_organizer.py

def __init__(self):
    """
        This class stores all variables required for analysis as well as
        methods to process it.
        Global variables (i.e. that does not concern the MotionAnalysis)
        are directly stored in self.
        Variables used in the MotionAnalysis class are stored in a dict
        called self.vars
    """
    self.one_arena_done: bool = False
    self.reduce_image_dim: bool = False
    self.first_exp_ready_to_run: bool = False
    self.sample_number = 1
    self.top = None
    self.motion = None
    self.analysis_instance = None
    self.computed_video_options = np.zeros(5, bool)
    self.vars = {}
    self.all = {}
    self.all['folder_list'] = []
    self.vars['first_detection_frame'] = 0
    self.first_im = None
    self.last_im = None
    self.starting_blob_hsize_in_pixels = None
    self.vars['first_move_threshold'] = 10
    self.vars['convert_for_origin'] = None
    self.vars['convert_for_motion'] = None
    self.current_combination_id = 0
    self.data_list = []
    self.one_row_per_arena = None
    self.one_row_per_frame = None
    self.not_analyzed_individuals = None
    self.bio_mask = None
    self.back_mask = None
    self.visualize: bool = True
    self.network_shaped: bool = False
    self.update_background_luminosity: bool = False

add_analysis_visualization_to_first_and_last_images(i, first_visualization, last_visualization=None)

Adds analysis visualizations to the first and last images of a sequence.

Parameters:

Name	Type	Description	Default
i	int	Index of the image in the sequence.	required
first_visualization	NDArray[uint8]	The visualization to add to the first image.	required
last_visualization	NDArray[uint8]	The visualization to add to the last image.	None

Other Parameters:

Name	Type	Description
vars	dict	Dictionary containing various parameters.
arena_shape	str	The shape of the arena. Either 'circle' or other shapes.

Notes

If arena_shape is 'circle', the visualization will be masked by an ellipse.

Source code in src/cellects/core/program_organizer.py

def add_analysis_visualization_to_first_and_last_images(self, i: int, first_visualization: NDArray, last_visualization: NDArray=None):
    """
    Adds analysis visualizations to the first and last images of a sequence.

    Parameters
    ----------
    i : int
        Index of the image in the sequence.
    first_visualization : NDArray[np.uint8]
        The visualization to add to the first image.
    last_visualization : NDArray[np.uint8]
        The visualization to add to the last image.

    Other Parameters
    ----------------
    vars : dict
        Dictionary containing various parameters.
    arena_shape : str, optional
        The shape of the arena. Either 'circle' or other shapes.

    Notes
    -----
    If `arena_shape` is 'circle', the visualization will be masked by an ellipse.

    """
    minmax = (self.top[i], self.bot[i], self.left[i], self.right[i])
    self.first_image.bgr = draw_img_with_mask(self.first_image.bgr, self.first_image.bgr.shape[:2], minmax,
                                              self.vars['arena_shape'], first_visualization)
    if last_visualization is not None:
        self.last_image.bgr = draw_img_with_mask(self.last_image.bgr, self.last_image.bgr.shape[:2], minmax,
                                                  self.vars['arena_shape'], last_visualization)

cropping(is_first_image)

Crops the image based on specified conditions and settings.

This method checks if drift correction has already been applied. If the image is the first one and hasn't been cropped yet, it will attempt to use pre-stored coordinates or compute new crop coordinates. If automatic cropping is enabled, it will apply the cropping process.

Parameters:

Name	Type	Description	Default
is_first_image	bool	Indicates whether the image being processed is the first one in the sequence.	required

Source code in src/cellects/core/program_organizer.py

def cropping(self, is_first_image: bool):
    """
    Crops the image based on specified conditions and settings.

    This method checks if drift correction has already been applied.
    If the image is the first one and hasn't been cropped yet, it will attempt
    to use pre-stored coordinates or compute new crop coordinates. If automatic
    cropping is enabled, it will apply the cropping process.

    Parameters
    ----------
    is_first_image : bool
        Indicates whether the image being processed is the first one in the sequence.
    """
    if not self.vars['drift_already_corrected'] and self.all['automatically_crop']:
        if is_first_image:
            if not self.first_image.cropped:
                self.first_image.get_crop_coordinates()
                self.first_image.automatically_crop(self.first_image.crop_coord)
        else:
            if not self.last_image.cropped:
                self.last_image.automatically_crop(self.first_image.crop_coord)

delineate_each_arena()

Determine the coordinates of each arena for video analysis.

The function processes video frames to identify bounding boxes around specimens and determines valid arenas for analysis. In case of existing data, it uses previously computed coordinates if available and valid.

Returns:

Name	Type	Description
analysis_status	dict	A dictionary containing flags and messages indicating the status of the analysis. - 'continue' (bool): Whether to continue processing. - 'message' (str): Informational or error message.

Notes

This function relies on the existence of certain attributes and variables defined in the class instance.

Source code in src/cellects/core/program_organizer.py

def delineate_each_arena(self):
    """
    Determine the coordinates of each arena for video analysis.

    The function processes video frames to identify bounding boxes around
    specimens and determines valid arenas for analysis. In case of existing data,
    it uses previously computed coordinates if available and valid.

    Returns
    -------
    analysis_status : dict
        A dictionary containing flags and messages indicating the status of
        the analysis.
        - 'continue' (bool): Whether to continue processing.
        - 'message' (str): Informational or error message.

    Notes
    -----
    This function relies on the existence of certain attributes and variables
    defined in the class instance.
    """
    analysis_status = {"continue": True, "message": ""}
    if not self.vars['several_blob_per_arena'] and (self.sample_number > 1):
        motion_list = None
        if self.all['are_gravity_centers_moving']:
            motion_list = self._segment_blob_motion(sample_size=5)
        self.get_bounding_boxes(are_gravity_centers_moving=self.all['are_gravity_centers_moving'] == 1,
            motion_list=motion_list, all_specimens_have_same_direction=self.all['all_specimens_have_same_direction'])

        if np.any(self.ordered_stats[:, 4] > 100 * np.median(self.ordered_stats[:, 4])):
            analysis_status['message'] = "A specimen is at least 100 times larger: click previous and retry by specifying 'back' areas."
            analysis_status['continue'] = False
        if np.any(self.ordered_stats[:, 4] < 0.01 * np.median(self.ordered_stats[:, 4])):
            analysis_status['message'] = "A specimen is at least 100 times smaller: click previous and retry by specifying 'back' areas."
            analysis_status['continue'] = False
        del self.ordered_stats
        logging.info(
            str(self.not_analyzed_individuals) + " individuals are out of picture scope and cannot be analyzed")
    else:
        self._whole_image_bounding_boxes()
        self.sample_number = 1
    self._set_analyzed_individuals()
    self.vars['arena_coord'] = []
    self.save_coordinates()
    return analysis_status

fast_first_image_segmentation()

Segment the first or subsequent image in a series for biological and background masks.

Notes

This function processes the first or subsequent image in a sequence, applying biological and background masks, segmenting the image, and updating internal data structures accordingly. The function is specific to handling image sequences for biological analysis

Source code in src/cellects/core/program_organizer.py

def fast_first_image_segmentation(self):
    """
    Segment the first or subsequent image in a series for biological and background masks.

    Notes
    -----
    This function processes the first or subsequent image in a sequence, applying biological and background masks,
    segmenting the image, and updating internal data structures accordingly. The function is specific to handling
    image sequences for biological analysis

    """
    if not "color_number" in self.vars:
        self.update_variable_dict()
    if self.vars['convert_for_origin'] is None:
        self.vars['convert_for_origin'] = {"logical": 'None', "PCA": [1, 1, 1]}
    self.first_image.convert_and_segment(self.vars['convert_for_origin'], self.vars["color_number"],
                                         self.all['initial_bio_mask'], self.all['initial_back_mask'], subtract_background=None,
                                         subtract_background2=None,
                                         rolling_window_segmentation=self.vars["rolling_window_segmentation"],
                                         filter_spec=self.vars["filter_spec"])
    if not self.first_image.drift_correction_already_adjusted:
        self.vars['drift_already_corrected'] = self.first_image.check_if_image_border_attest_drift_correction()
        if self.vars['drift_already_corrected']:
            logging.info("Cellects detected that the images have already been corrected for drift")
            self.first_image.convert_and_segment(self.vars['convert_for_origin'], self.vars["color_number"],
                                                 self.all['initial_bio_mask'], self.all['initial_back_mask'],
                                                 subtract_background=None, subtract_background2=None,
                                                 rolling_window_segmentation=self.vars["rolling_window_segmentation"],
                                                 filter_spec=self.vars["filter_spec"],
                                                 allowed_window=self.first_image.drift_mask_coord)

    shapes_features = shape_selection(self.first_image.binary_image, true_shape_number=self.sample_number,
                                      horizontal_size=self.starting_blob_hsize_in_pixels,
                                      spot_shape=self.all['starting_blob_shape'],
                                      several_blob_per_arena=self.vars['several_blob_per_arena'],
                                      bio_mask=self.all['initial_bio_mask'], back_mask=self.all['initial_back_mask'])
    self.first_image.validated_shapes, shape_number, stats, centroids = shapes_features
    self.first_image.shape_number = shape_number
    if self.first_image.im_combinations is None:
        self.first_image.im_combinations = []
    if len(self.first_image.im_combinations) == 0:
        self.first_image.im_combinations.append({})
    self.current_combination_id = np.min((self.current_combination_id, len(self.first_image.im_combinations) - 1))
    self.first_image.im_combinations[self.current_combination_id]['csc'] = self.vars['convert_for_origin']
    self.first_image.im_combinations[self.current_combination_id]['binary_image'] = self.first_image.validated_shapes
    if self.first_image.greyscale is not None:
        greyscale = self.first_image.greyscale
    else:
        greyscale = self.first_image.image
    self.first_image.im_combinations[self.current_combination_id]['converted_image'] = bracket_to_uint8_image_contrast(greyscale)
    self.first_image.im_combinations[self.current_combination_id]['shape_number'] = shape_number

fast_last_image_segmentation(bio_mask=None, back_mask=None)

Segment the first or subsequent image in a series for biological and background masks.

Parameters:

Name	Type	Description	Default
bio_mask	NDArray[uint8]	The biological mask to be applied to the image.	None
back_mask	NDArray[uint8]	The background mask to be applied to the image.	None

Returns:

Type	Description
None

Notes

This function processes the first or subsequent image in a sequence, applying biological and background masks, segmenting the image, and updating internal data structures accordingly. The function is specific to handling image sequences for biological analysis

Source code in src/cellects/core/program_organizer.py

def fast_last_image_segmentation(self, bio_mask: NDArray[np.uint8] = None, back_mask: NDArray[np.uint8] = None):
    """
    Segment the first or subsequent image in a series for biological and background masks.

    Parameters
    ----------
    bio_mask : NDArray[np.uint8], optional
        The biological mask to be applied to the image.
    back_mask : NDArray[np.uint8], optional
        The background mask to be applied to the image.

    Returns
    -------
    None

    Notes
    -----
    This function processes the first or subsequent image in a sequence, applying biological and background masks,
    segmenting the image, and updating internal data structures accordingly. The function is specific to handling
    image sequences for biological analysis

    """
    if self.vars['convert_for_motion'] is None:
        self.vars['convert_for_motion'] = {"logical": 'None', "PCA": [1, 1, 1]}
    self.cropping(is_first_image=False)
    self.last_image.convert_and_segment(self.vars['convert_for_motion'], self.vars["color_number"],
                                        bio_mask, back_mask, self.first_image.subtract_background,
                                        self.first_image.subtract_background2,
                                        rolling_window_segmentation=self.vars["rolling_window_segmentation"],
                                        filter_spec=self.vars["filter_spec"])
    if self.vars['drift_already_corrected'] and not self.last_image.drift_correction_already_adjusted and not self.vars["rolling_window_segmentation"]['do']:
        self.last_image.check_if_image_border_attest_drift_correction()
        self.last_image.convert_and_segment(self.vars['convert_for_motion'], self.vars["color_number"],
                                            bio_mask, back_mask, self.first_image.subtract_background,
                                            self.first_image.subtract_background2,
                                            allowed_window=self.last_image.drift_mask_coord,
                                            filter_spec=self.vars["filter_spec"])

    if self.last_image.im_combinations is None:
        self.last_image.im_combinations = []
    if len(self.last_image.im_combinations) == 0:
        self.last_image.im_combinations.append({})
    self.current_combination_id = np.min((self.current_combination_id, len(self.last_image.im_combinations) - 1))
    self.last_image.im_combinations[self.current_combination_id]['csc'] = self.vars['convert_for_motion']
    self.last_image.im_combinations[self.current_combination_id]['binary_image'] = self.last_image.binary_image
    if self.last_image.greyscale is not None:
        greyscale = self.last_image.greyscale
    else:
        greyscale = self.last_image.image
    self.last_image.im_combinations[self.current_combination_id]['converted_image'] = bracket_to_uint8_image_contrast(greyscale)

find_if_lighter_background()

Determines whether the background is lighter or darker than the cells.

This function analyzes images to determine if their backgrounds are lighter or darker relative to the cells, updating attributes accordingly for analysis and display purposes.

Notes

This function modifies instance variables and does not return any value. The analysis involves comparing mean pixel values in specific areas of the image.

Source code in src/cellects/core/program_organizer.py

def find_if_lighter_background(self):
    """
    Determines whether the background is lighter or darker than the cells.

    This function analyzes images to determine if their backgrounds are lighter
    or darker relative to the cells, updating attributes accordingly for analysis and display purposes.


    Notes
    -----
    This function modifies instance variables and does not return any value.
    The analysis involves comparing mean pixel values in specific areas of the image.
    """
    logging.info("Find if the background is lighter or darker than the cells")
    self.vars['lighter_background']: bool = True
    self.vars['contour_color']: np.uint8 = 0
    are_dicts_equal: bool = True
    if self.vars['convert_for_origin'] is not None and self.vars['convert_for_origin'] is not None:
        for key in self.vars['convert_for_origin'].keys():
            are_dicts_equal = are_dicts_equal and np.all(key in self.vars['convert_for_motion'] and self.vars['convert_for_origin'][key] == self.vars['convert_for_motion'][key])

        for key in self.vars['convert_for_motion'].keys():
            are_dicts_equal = are_dicts_equal and np.all(key in self.vars['convert_for_origin'] and self.vars['convert_for_motion'][key] == self.vars['convert_for_origin'][key])
    else:
        self.vars['convert_for_origin'] = {"logical": 'None', "PCA": [1, 1, 1]}
    if are_dicts_equal:
        if self.first_im is None:
            self.get_first_image()
            self.fast_first_image_segmentation()
            self.cropping(is_first_image=True)
        among = np.nonzero(self.first_image.validated_shapes)
        not_among = np.nonzero(1 - self.first_image.validated_shapes)
        # Use the converted image to tell if the background is lighter, for analysis purposes
        if self.first_image.image[among[0], among[1]].mean() > self.first_image.image[not_among[0], not_among[1]].mean():
            self.vars['lighter_background'] = False
        # Use the original image to tell if the background is lighter, for display purposes
        if self.first_image.bgr[among[0], among[1], ...].mean() > self.first_image.bgr[not_among[0], not_among[1], ...].mean():
            self.vars['contour_color'] = 255
    else:
        if self.last_im is None:
            self.get_last_image()
            # self.cropping(is_first_image=False)
            self.fast_last_image_segmentation()
        if self.last_image.binary_image.sum() == 0:
            self.fast_last_image_segmentation()
        among = np.nonzero(self.last_image.binary_image)
        not_among = np.nonzero(1 - self.last_image.binary_image)
        # Use the converted image to tell if the background is lighter, for analysis purposes
        if self.last_image.image[among[0], among[1]].mean() > self.last_image.image[not_among[0], not_among[1]].mean():
            self.vars['lighter_background'] = False
        # Use the original image to tell if the background is lighter, for display purposes
        if self.last_image.bgr[among[0], among[1], ...].mean() > self.last_image.bgr[not_among[0], not_among[1], ...].mean():
            self.vars['contour_color'] = 255
    if self.vars['origin_state'] == "invisible":
        binary_image = self.first_image.binary_image.copy()
        self.first_image.convert_and_segment(self.vars['convert_for_motion'], self.vars["color_number"],
                                             None, None, subtract_background=None,
                                             subtract_background2=None,
                                             rolling_window_segmentation=self.vars['rolling_window_segmentation'],
                                             filter_spec=self.vars["filter_spec"])
        covered_values = self.first_image.image[np.nonzero(binary_image)]
        self.vars['luminosity_threshold'] = 127
        if len(covered_values) > 0:
            if self.vars['lighter_background']:
                if np.max(covered_values) < 255:
                    self.vars['luminosity_threshold'] = np.max(covered_values) + 1
            else:
                if np.min(covered_values) > 0:
                    self.vars['luminosity_threshold'] = np.min(covered_values) - 1

get_average_pixel_size()

Calculate the average pixel size and related variables.

Logs information about calculation steps, computes the average pixel size based on image or cell scaling settings, and sets initial thresholds for object detection.

Notes

• The average pixel size is determined by either image dimensions or blob sizes.
• Thresholds for automatic detection are set based on configuration settings.

Source code in src/cellects/core/program_organizer.py

def get_average_pixel_size(self):
    """
    Calculate the average pixel size and related variables.

    Logs information about calculation steps, computes the average
    pixel size based on image or cell scaling settings,
    and sets initial thresholds for object detection.

    Notes
    -----
    - The average pixel size is determined by either image dimensions or blob sizes.
    - Thresholds for automatic detection are set based on configuration settings.

    """
    logging.info("Getting average pixel size")
    (self.first_image.shape_number,
        self.first_image.shapes,
        self.first_image.stats,
        centroids) = cv2.connectedComponentsWithStats(
            self.first_image.validated_shapes,
            connectivity=8)
    self.first_image.shape_number -= 1
    self.vars['average_pixel_size']: float = 1.
    if self.all['scale_with_image_or_cells'] == 0:
        self.vars['average_pixel_size'] = float(np.square(self.all['image_horizontal_size_in_mm'] /
                                                    self.first_im.shape[1]))
    else:
        if len(self.first_image.stats[1:, 2]) > 0:
            self.vars['average_pixel_size'] = float(np.square(self.all['starting_blob_hsize_in_mm'] /
                                                        np.mean(self.first_image.stats[1:, 2])))
        else:
            self.vars['output_in_mm'] = False

    if self.all['set_spot_size']:
        self.starting_blob_hsize_in_pixels = float((self.all['starting_blob_hsize_in_mm'] /
                                              np.sqrt(self.vars['average_pixel_size'])))
    else:
        self.starting_blob_hsize_in_pixels = None

    if not self.all['automatic_size_thresholding']:
        self.vars['first_move_threshold'] = int(np.round(self.all['first_move_threshold_in_mm²'] /
                                                     self.vars['average_pixel_size']))
    logging.info(f"The average pixel size is: {self.vars['average_pixel_size']} mm²")

get_background_to_subtract()

Determine if background subtraction should be applied to the image.

Extended Description

This function checks whether background subtraction should be applied. It utilizes the 'subtract_background' flag and potentially converts the image for motion estimation.

Parameters:

Name	Type	Description	Default
self	object	The instance of the class containing this method. Must have attributes vars and first_image.	required

Source code in src/cellects/core/program_organizer.py

def get_background_to_subtract(self):
    """
    Determine if background subtraction should be applied to the image.

    Extended Description
    --------------------
    This function checks whether background subtraction should be applied.
    It utilizes the 'subtract_background' flag and potentially converts
    the image for motion estimation.

    Parameters
    ----------
    self : object
        The instance of the class containing this method.
        Must have attributes `vars` and `first_image`.
    """
    if self.vars['subtract_background']:
        self.first_image.generate_subtract_background(self.vars['convert_for_motion'], self.vars['drift_already_corrected'])

get_bounding_boxes(are_gravity_centers_moving, motion_list=(), all_specimens_have_same_direction=True, original_shape_hsize=None)

Get the coordinates of arenas using bounding boxes.

Parameters:

Name	Type	Description	Default
are_gravity_centers_moving	bool	Flag indicating whether gravity centers are moving or not.	required
motion_list	list	List of motion information for the specimens.	()
all_specimens_have_same_direction	bool	Flag indicating whether all specimens have the same direction, by default True.	True

Notes

This method uses various internal methods and variables to determine the bounding boxes.

Source code in src/cellects/core/program_organizer.py

def get_bounding_boxes(self, are_gravity_centers_moving: bool, motion_list: list=(), all_specimens_have_same_direction: bool=True, original_shape_hsize: int=None):
    """Get the coordinates of arenas using bounding boxes.

    Parameters
    ----------
    are_gravity_centers_moving : bool
        Flag indicating whether gravity centers are moving or not.
    motion_list : list
        List of motion information for the specimens.
    all_specimens_have_same_direction : bool, optional
        Flag indicating whether all specimens have the same direction,
        by default True.
    Notes
    -----
    This method uses various internal methods and variables to determine the bounding boxes.
    """
    # 7) Create required empty arrays: especially the bounding box coordinates of each video
    self.ordered_first_image = None
    self.shapes_to_remove = None
    if self.first_image.y_boundaries is None:
        self.first_image.get_setup_boundaries()

    logging.info("Get the coordinates of all arenas using the get_bounding_boxes method")
    if self.first_image.validated_shapes.any() and self.first_image.shape_number > 0:
        self.ordered_stats, ordered_centroids, self.ordered_first_image = rank_from_top_to_bottom_from_left_to_right(
            self.first_image.validated_shapes, self.first_image.y_boundaries, get_ordered_image=True)
        self.unchanged_ordered_fimg = self.ordered_first_image.copy()
        self.modif_validated_shapes = self.first_image.validated_shapes.copy()
        self.standard = - 1
        counter = 0
        while np.any(np.less(self.standard, 0)) and counter < 20:
            counter += 1
            self.left = np.zeros(self.first_image.shape_number, dtype=np.int64)
            self.right = np.repeat(self.modif_validated_shapes.shape[1], self.first_image.shape_number)
            self.top = np.zeros(self.first_image.shape_number, dtype=np.int64)
            self.bot = np.repeat(self.modif_validated_shapes.shape[0], self.first_image.shape_number)
            if are_gravity_centers_moving:
                self.top, self.bot, self.left, self.right, self.ordered_first_image = get_bb_with_moving_centers(motion_list, all_specimens_have_same_direction,
                                                 original_shape_hsize, self.first_image.validated_shapes,
                                                 self.first_image.y_boundaries)
                new_ordered_first_image = np.zeros(self.ordered_first_image.shape, dtype=np.uint8)

                for i in np.arange(1, self.first_image.shape_number + 1):
                    previous_shape = np.zeros(self.ordered_first_image.shape, dtype=np.uint8)
                    previous_shape[np.nonzero(self.unchanged_ordered_fimg == i)] = 1
                    new_potentials = np.zeros(self.ordered_first_image.shape, dtype=np.uint8)
                    new_potentials[np.nonzero(self.ordered_first_image == i)] = 1
                    new_potentials[np.nonzero(self.unchanged_ordered_fimg == i)] = 0

                    pads = ProgressivelyAddDistantShapes(new_potentials, previous_shape, max_distance=2)
                    pads.consider_shapes_sizes(min_shape_size=10)
                    pads.connect_shapes(only_keep_connected_shapes=True, rank_connecting_pixels=False)
                    new_ordered_first_image[np.nonzero(pads.expanded_shape)] = i
                self.ordered_first_image = new_ordered_first_image
                self.modif_validated_shapes = np.zeros(self.ordered_first_image.shape, dtype=np.uint8)
                self.modif_validated_shapes[np.nonzero(self.ordered_first_image)] = 1
                self.ordered_stats, ordered_centroids, self.ordered_first_image = rank_from_top_to_bottom_from_left_to_right(
                    self.modif_validated_shapes, self.first_image.y_boundaries, get_ordered_image=True)
                self.top, self.bot, self.left, self.right = get_quick_bounding_boxes(self.modif_validated_shapes, self.ordered_first_image, self.ordered_stats)
            else:
                self.top, self.bot, self.left, self.right = get_quick_bounding_boxes(self.modif_validated_shapes, self.ordered_first_image, self.ordered_stats)
            self._standardize_video_sizes()
        if counter == 20:
            self.top[self.top < 0] = 1
            self.bot[self.bot >= self.ordered_first_image.shape[0] - 1] = self.ordered_first_image.shape[0] - 1
            self.left[self.left < 0] = 1
            self.right[self.right >= self.ordered_first_image.shape[1] - 1] = self.ordered_first_image.shape[1] - 1
        del self.ordered_first_image
        del self.unchanged_ordered_fimg
        del self.modif_validated_shapes
        del self.standard
        del self.shapes_to_remove
        self.bot += 1
        self.right += 1
    else:
        self._whole_image_bounding_boxes()

get_first_image(first_im=None, sample_number=None)

Load and process the first image or frame from a video.

This method handles loading the first image or the first frame of a video depending on whether the data is an image or a video. It performs necessary preprocessing and initializes relevant attributes for subsequent analysis.

Source code in src/cellects/core/program_organizer.py

def get_first_image(self, first_im: NDArray=None, sample_number: int=None):
    """
    Load and process the first image or frame from a video.

    This method handles loading the first image or the first frame of a video
    depending on whether the data is an image or a video. It performs necessary
    preprocessing and initializes relevant attributes for subsequent analysis.
    """
    if sample_number is not None:
        self.sample_number = sample_number
    self.reduce_image_dim = False
    if first_im is not None:
        self.first_im = first_im
    else:
        logging.info("Load first image")
        if self.all['im_or_vid'] == 1:
            if self.analysis_instance is None:
                self.analysis_instance = video2numpy(self.data_list[0])
                self.sample_number = len(self.data_list)
                self.vars['img_number'] = self.analysis_instance.shape[0]
                self.first_im = self.analysis_instance[0, ...]
                self.vars['dims'] = self.analysis_instance.shape[:3]
            else:
                self.first_im = self.analysis_instance[self.vars['first_detection_frame'], ...]

        else:
            self.vars['img_number'] = len(self.data_list)
            self.all['raw_images'] = is_raw_image(self.data_list[0])
            self.first_im = readim(self.data_list[self.vars['first_detection_frame']], self.all['raw_images'])
            self.vars['dims'] = [self.vars['img_number'], self.first_im.shape[0], self.first_im.shape[1]]

            if len(self.first_im.shape) == 3:
                if np.all(np.equal(self.first_im[:, :, 0], self.first_im[:, :, 1])) and np.all(
                        np.equal(self.first_im[:, :, 1], self.first_im[:, :, 2])):
                    self.reduce_image_dim = True
                if self.reduce_image_dim:
                    self.first_im = self.first_im[:, :, 0]

    self.first_image = OneImageAnalysis(self.first_im, self.sample_number)
    self.vars['already_greyscale'] = self.first_image.already_greyscale
    if self.vars['already_greyscale']:
        self.vars["convert_for_origin"] = {"bgr": [1, 1, 1], "logical": "None"}
        self.vars["convert_for_motion"] = {"bgr": [1, 1, 1], "logical": "None"}
    if np.mean((np.mean(self.first_image.image[2, :, ...]), np.mean(self.first_image.image[-3, :, ...]), np.mean(self.first_image.image[:, 2, ...]), np.mean(self.first_image.image[:, -3, ...]))) > 127:
        self.vars['contour_color']: np.uint8 = 0
    else:
        self.vars['contour_color']: np.uint8 = 255
    if self.vars['first_detection_frame'] > 0:
        self.vars['origin_state'] = 'invisible'

get_last_image(last_im=None)

Load the last image from a video or image list and process it based on given parameters.

Parameters:

Name	Type	Description	Default
last_im	NDArray	The last image to be loaded. If not provided, the last image will be loaded from the data list.	None

Source code in src/cellects/core/program_organizer.py

def get_last_image(self, last_im: NDArray=None):
    """

    Load the last image from a video or image list and process it based on given parameters.

    Parameters
    ----------
    last_im : NDArray, optional
        The last image to be loaded. If not provided, the last image will be loaded from the data list.
    """
    logging.info("Load last image")
    if last_im is not None:
        self.last_im = last_im
    else:
        if self.all['im_or_vid'] == 1:
            self.last_im = self.analysis_instance[-1, ...]
        else:
            is_landscape = self.first_image.image.shape[0] < self.first_image.image.shape[1]
            self.last_im = read_and_rotate(self.data_list[-1], self.first_im, self.all['raw_images'], is_landscape)
            if self.reduce_image_dim:
                self.last_im = self.last_im[:, :, 0]
    self.last_image = OneImageAnalysis(self.last_im)

instantiate_tables()

Update output list and prepare results tables and validation images.

Extended Description

This method performs necessary preparations for processing image sequences, including updating the output list and initializing key attributes required for subsequent operations.

Source code in src/cellects/core/program_organizer.py

def instantiate_tables(self):
    """
    Update output list and prepare results tables and validation images.

    Extended Description
    --------------------
    This method performs necessary preparations for processing image sequences,
    including updating the output list and initializing key attributes required
    for subsequent operations.

    """
    self.update_output_list()
    logging.info("Instantiate results tables and validation images")
    self.fractal_box_sizes = None
    self.one_row_per_arena = None
    self.one_row_per_frame = None
    if self.vars['already_greyscale']:
        if len(self.first_image.bgr.shape) == 2:
            self.first_image.bgr = np.stack((self.first_image.bgr, self.first_image.bgr, self.first_image.bgr), axis=2).astype(np.uint8)
        if len(self.last_image.bgr.shape) == 2:
            self.last_image.bgr = np.stack((self.last_image.bgr, self.last_image.bgr, self.last_image.bgr), axis=2).astype(np.uint8)
        self.vars["convert_for_motion"] = {"bgr": [1, 1, 1], "logical": "None"}

load_data_to_run_cellects_quickly()

Load data from a pickle file and update the current state of the object.

Summarizes, loads, and validates data needed to run Cellects, updating the object's state accordingly. If the necessary data are not present or valid, it ensures the experiment is marked as not ready to run.

Parameters:

Name	Type	Description	Default
self	CellectsObject	The instance of the class (assumed to be a subclass of CellectsObject) that this method belongs to.	required

Returns:

Type	Description
None

Notes

This function relies on the presence of a pickle file 'cellects_settings.json'. It updates the state of various attributes based on the loaded data and logs appropriate messages.

Source code in src/cellects/core/program_organizer.py

def load_data_to_run_cellects_quickly(self):
    """
    Load data from a pickle file and update the current state of the object.

    Summarizes, loads, and validates data needed to run Cellects,
    updating the object's state accordingly. If the necessary data
    are not present or valid, it ensures the experiment is marked as
    not ready to run.

    Parameters
    ----------
    self : CellectsObject
        The instance of the class (assumed to be a subclass of
        CellectsObject) that this method belongs to.

    Returns
    -------
    None

    Notes
    -----
    This function relies on the presence of a pickle file 'cellects_settings.json'.
    It updates the state of various attributes based on the loaded data
    and logs appropriate messages.
    """
    self.analysis_instance = None
    self.first_im = None
    self.first_image = None
    self.last_image = None
    current_global_pathway = self.all['global_pathway']
    folder_number = self.all['folder_number']
    if folder_number > 1:
        folder_list = self.all['folder_list'].copy()
        sample_number_per_folder = self.all['sample_number_per_folder'].copy()

    self.first_exp_ready_to_run: bool = False
    if os.path.isfile('cellects_settings.json'):
        data_to_run_cellects_quickly = read_json('cellects_settings.json')
        if data_to_run_cellects_quickly is None:
            data_to_run_cellects_quickly = {}
        if (os.path.isfile('ind_1.h5')) and (os.path.isfile('cellects_data.h5')) and ('all' in data_to_run_cellects_quickly):
            ind1_keys = get_h5_keys('ind_1.h5')
            cellects_data_keys = get_h5_keys('cellects_data.h5')
            if 'origin_coord' in ind1_keys and 'arenas_coord' in cellects_data_keys and 'exif' in cellects_data_keys:
                logging.info("Success to load cellects_settings.json from the user chosen directory")
                self.all = data_to_run_cellects_quickly['all']
                # If you want to add a new variable, first run an updated version of all_vars_dict,
                # then put a breakpoint here and run the following + self.save_data_to_run_cellects_quickly() :
                self.vars = self.all['vars']
                self.update_variable_dict()
                folder_changed = False
                if current_global_pathway != self.all['global_pathway']:
                    folder_changed = True
                    logging.info("Although the folder is ready, it is not at the same place as it was during creation, updating")
                    self.all['global_pathway'] = current_global_pathway
                if folder_number > 1:
                    self.all['global_pathway'] = current_global_pathway
                    self.all['folder_list'] = folder_list
                    self.all['folder_number'] = folder_number
                    self.all['sample_number_per_folder'] = sample_number_per_folder
                    self.all['first_folder_sample_number'] = sample_number_per_folder[0]

                if len(self.data_list) == 0:
                    self.look_for_data()
                    if folder_changed and folder_number > 1 and len(self.all['folder_list']) > 0:
                        self.update_folder_id(self.all['sample_number_per_folder'][0], self.all['folder_list'][0])
                if len(self.data_list) > 0:
                    self.get_first_image()
                    self.get_last_image()
                    self.top, self.bot, self.left, self.right = read_h5('cellects_data.h5', 'arenas_coord')
                    self.vars['arenas_coord'] = [self.top, self.bot, self.left, self.right]
                    self.vars['exif'] = read_h5('cellects_data.h5', 'exif')
                    self.vars['crop_coord'] = None
                    if self.all['automatically_crop'] and 'crop_coord' in cellects_data_keys:
                        ccy1, ccy2, ccx1, ccx2 = read_h5('cellects_data.h5', 'crop_coord')
                        self.first_image.crop_coord = [ccy1, ccy2, ccx1, ccx2]
                        self.vars['crop_coord'] = self.first_image.crop_coord
                        logging.info("Crop first image")
                        self.first_image.automatically_crop(self.first_image.crop_coord)
                        logging.info("Crop last image")
                        self.last_image.automatically_crop(self.first_image.crop_coord)
                    shapes_coord = read_h5('cellects_data.h5','validated_shapes')
                    if shapes_coord is not None:
                        self.first_image.validated_shapes = np.zeros(self.first_image.image.shape[:2], np.uint8)
                        self.first_image.validated_shapes[shapes_coord[0], shapes_coord[1]] = 1
                        self.first_image.im_combinations = []
                        self.current_combination_id = 0
                        self.first_image.im_combinations.append({})
                        self.first_image.im_combinations[self.current_combination_id]['csc'] = self.vars['convert_for_origin']
                        self.first_image.im_combinations[self.current_combination_id]['binary_image'] = self.first_image.validated_shapes
                        self.first_image.im_combinations[self.current_combination_id]['shape_number'] = data_to_run_cellects_quickly['shape_number']
                        if not 'average_pixel_size' in self.vars:
                            self.get_average_pixel_size()
                        if not 'lighter_background' in self.vars:
                            self.find_if_lighter_background()
                        background = read_h5(f'ind_{1}.h5', 'background')
                        if not self.vars['subtract_background'] or (self.vars['subtract_background'] and background is not None):
                            self.first_exp_ready_to_run = True
    if self.first_exp_ready_to_run:
        logging.info("The current folder is ready to run")
    else:
        logging.info("The current folder is not ready to run")

load_masks()

Source code in src/cellects/core/program_organizer.py

def load_masks(self):
    """"""
    if self.all['keep_cell_and_back_for_all_folders']:
        self.bio_mask = read_h5(CONFIG_DIR / 'masks.h5', 'initial_bio_mask')
        self.back_mask = read_h5(CONFIG_DIR / 'masks.h5', 'initial_back_mask')

load_variable_dict()

Loads configuration dictionaries from a pickle file if available, otherwise initializes defaults.

Tries to load saved parameters. If the file doesn't exist or loading fails due to corruption, default values are used instead (logging relevant warnings).

Raises:

Type	Description
FileNotFoundError	If no valid configuration file is found and default initialization fails.

Notes

This method ensures robust operation by handling missing or corrupted configuration files gracefully.

Source code in src/cellects/core/program_organizer.py

def load_variable_dict(self):
    """
    Loads configuration dictionaries from a pickle file if available, otherwise initializes defaults.

    Tries to load saved parameters. If the file doesn't exist or loading fails due to corruption,
    default values are used instead (logging relevant warnings).

    Raises
    ------
    FileNotFoundError
        If no valid configuration file is found and default initialization fails.

    Notes
    -----
    This method ensures robust operation by handling missing or corrupted configuration files gracefully.
    """
    if os.path.isfile(ALL_VARS_JSON_FILE):
        logging.info("Load the parameters from cellects_settings.json in the config of the Cellects folder")
        try:
            self.all = read_json(ALL_VARS_JSON_FILE)
            self.vars = self.all['vars']
            self.update_variable_dict()
            logging.info("Success to load the parameters dictionaries from the Cellects folder")
        except Exception as exc:
            logging.error(f"Initialize default parameters because error: {exc}")
            default_dicts = DefaultDicts()
            self.all = default_dicts.all
            self.vars = default_dicts.vars
    else:
        logging.info("Initialize default parameters")
        default_dicts = DefaultDicts()
        self.all = default_dicts.all
        self.vars = default_dicts.vars
    if self.all['cores'] == 1:
        self.all['cores'] = os.cpu_count() - 1

look_for_data()

Discovers all relevant video/image data in the working directory.

Uses natural sorting to handle filenames with numeric suffixes. Validates file consistency and logs warnings if filename patterns are inconsistent across folders.

Raises:

Type	Description
ValueError	If no files match the specified naming convention.

Notes

This method assumes all data files follow a predictable pattern with numeric extensions. Use caution in unpredictable directory structures where this may fail silently or produce incorrect results.

Examples:

>>> organizer.look_for_data()
>>> print(organizer.data_list)
['/path/to/video1.avi', '/path/to/video2.avi']

Source code in src/cellects/core/program_organizer.py

def look_for_data(self):
    """
    Discovers all relevant video/image data in the working directory.

    Uses natural sorting to handle filenames with numeric suffixes. Validates file consistency and logs warnings
    if filename patterns are inconsistent across folders.

    Raises
    ------
    ValueError
        If no files match the specified naming convention.

    Notes
    -----
    This method assumes all data files follow a predictable pattern with numeric extensions. Use caution in
    unpredictable directory structures where this may fail silently or produce incorrect results.

    Examples
    --------
    >>> organizer.look_for_data()
    >>> print(organizer.data_list)
    ['/path/to/video1.avi', '/path/to/video2.avi']
    """
    os.chdir(Path(self.all['global_pathway']))
    logging.info(f"Dir: {self.all['global_pathway']}")
    self.data_list = insensitive_glob(self.all['radical'] + '*' + self.all['extension'])  # Provides a list ordered by last modification date
    self.all['folder_list'] = []
    self.all['folder_number'] = 1
    self.vars['first_detection_frame'] = 0
    if len(self.data_list) > 0:
        self._sort_data_list()
        self.sample_number = self.all['first_folder_sample_number']
    else:
        content = os.listdir()
        for obj in content:
            if not os.path.isfile(obj):
                data_list = insensitive_glob(obj + "/" + self.all['radical'] + '*' + self.all['extension'])
                if len(data_list) > 0:
                    self.all['folder_list'].append(obj)
                    self.all['folder_number'] += 1
        self.all['folder_list'] = np.sort(self.all['folder_list']).tolist()

        if isinstance(self.all['sample_number_per_folder'], int) or len(self.all['sample_number_per_folder']) == 1:
            self.all['sample_number_per_folder'] = np.repeat(self.all['sample_number_per_folder'], self.all['folder_number']).tolist()

prepare_video_writing(img_list, min_ram_free, in_colors=False, pathway='')

Prepare the raw video (.h5) writing process for Cellects.

Parameters:

Name	Type	Description	Default
img_list	list	List of images to be processed.	required
min_ram_free	float	Minimum amount of RAM in GB that should remain free.	required
in_colors	bool	Whether the images are in color. Default is False.	False
pathway	str	Path to save the video files. Default is an empty string.	''

Returns:

Type

Description

tuple

A tuple containing: - bunch_nb: int, number of bunches needed for video writing. - video_nb_per_bunch: int, number of videos per bunch. - sizes: ndarray, dimensions of each video. - video_bunch: list or ndarray, initialized video arrays. - vid_names: list, names of the video files. - rom_memory_required: None or float, required ROM memory. - analysis_status: dict, status and message of the analysis process. - remaining: int, remainder videos that do not fit in a complete bunch.

Notes

• The function calculates necessary memory and ensures 10% extra to avoid issues.
• It checks for available RAM and adjusts the number of bunches accordingly.
• If using color images, memory requirements are tripled.

expected output depends on the provided images and RAM availability

Source code in src/cellects/core/program_organizer.py

def prepare_video_writing(self, img_list: list, min_ram_free: float, in_colors: bool=False, pathway: str=""):
    """

    Prepare the raw video (.h5) writing process for Cellects.

    Parameters
    ----------
    img_list : list
        List of images to be processed.
    min_ram_free : float
        Minimum amount of RAM in GB that should remain free.
    in_colors : bool, optional
        Whether the images are in color. Default is False.
    pathway : str, optional
        Path to save the video files. Default is an empty string.

    Returns
    -------
    tuple
        A tuple containing:
        - bunch_nb: int, number of bunches needed for video writing.
        - video_nb_per_bunch: int, number of videos per bunch.
        - sizes: ndarray, dimensions of each video.
        - video_bunch: list or ndarray, initialized video arrays.
        - vid_names: list, names of the video files.
        - rom_memory_required: None or float, required ROM memory.
        - analysis_status: dict, status and message of the analysis process.
        - remaining: int, remainder videos that do not fit in a complete bunch.

    Notes
    -----
    - The function calculates necessary memory and ensures 10% extra to avoid issues.
    - It checks for available RAM and adjusts the number of bunches accordingly.
    - If using color images, memory requirements are tripled.

    expected output depends on the provided images and RAM availability
    """
    # 1) Create a list of video names
    if self.not_analyzed_individuals is not None:
        number_to_add = len(self.not_analyzed_individuals)
    else:
        number_to_add = 0
    vid_names = list()
    ind_i = 0
    counter = 0
    while ind_i < (self.first_image.shape_number + number_to_add):
        ind_i += 1
        while np.any(np.isin(self.not_analyzed_individuals, ind_i)):
            ind_i += 1
        vid_names.append(pathway + "ind_" + str(ind_i) + ".h5")
        counter += 1
    img_nb = len(img_list)

    # 2) Create a table of the dimensions of each video
    # Add 10% to the necessary memory to avoid problems
    vertical_diffs = np.array(self.bot) - np.array(self.top)
    horizontal_diffs = np.array(self.right) - np.array(self.left)
    necessary_memory = img_nb * np.multiply((vertical_diffs).astype(np.uint64), (horizontal_diffs).astype(np.uint64)).sum() * 8 * 1.16415e-10
    if in_colors:
        sizes = np.column_stack(
            (np.repeat(img_nb, self.first_image.shape_number), vertical_diffs, horizontal_diffs,
             np.repeat(3, self.first_image.shape_number)))
        necessary_memory *= 3
    else:
        sizes = np.column_stack(
            (np.repeat(img_nb, self.first_image.shape_number), vertical_diffs, horizontal_diffs))
    use_list_of_vid = True
    if np.all(sizes[0, :] == sizes):
        use_list_of_vid = False
    available_memory = (psutil.virtual_memory().available >> 30) - min_ram_free
    if available_memory == 0:
        analysis_status = {"continue": False, "message": "There are not enough RAM available"}
        bunch_nb = 1
    else:
        bunch_nb = int(np.ceil(necessary_memory / available_memory))
        if bunch_nb > 1:
            # The program will need twice the memory to create the second bunch.
            bunch_nb = int(np.ceil(2 * necessary_memory / available_memory))

    video_nb_per_bunch = np.floor(self.first_image.shape_number / bunch_nb).astype(np.uint8)
    analysis_status = {"continue": True, "message": ""}
    video_bunch = None
    try:
        if use_list_of_vid:
            video_bunch = [np.zeros(sizes[i, :], dtype=np.uint8) for i in range(video_nb_per_bunch)]
        else:
            video_bunch = np.zeros(np.append(sizes[0, :], video_nb_per_bunch), dtype=np.uint8)
    except ValueError as v_err:
        analysis_status = {"continue": False, "message": "Probably failed to detect the right cell(s) number, do the first image analysis manually."}
        logging.error(f"{analysis_status['message']} error is: {v_err}")
    # Check for available ROM memory
    if (psutil.disk_usage('/')[2] >> 30) < (necessary_memory + 2):
        rom_memory_required = necessary_memory + 2
    else:
        rom_memory_required = None
    remaining = self.first_image.shape_number % bunch_nb
    if remaining > 0:
        bunch_nb += 1
    is_landscape = self.first_image.image.shape[0] < self.first_image.image.shape[1]
    logging.info(f"Cellects will start writing {self.first_image.shape_number} videos. Given available memory, it will do it in {bunch_nb} time(s)")
    return bunch_nb, video_nb_per_bunch, sizes, video_bunch, vid_names, rom_memory_required, analysis_status, remaining, use_list_of_vid, is_landscape

save_coordinates()

Summarize the coordinates of images and video.

Combine the crop coordinates from the first image with additional coordinates for left, right, top, and bottom boundaries to form a list of video coordinates. If the crop coordinates are not already set, initialize them to cover the entire image.

Returns:

Type	Description
list of int	A list containing the coordinates [left, right, top, bottom] for video.

Source code in src/cellects/core/program_organizer.py

def save_coordinates(self):
    """
    Summarize the coordinates of images and video.

    Combine the crop coordinates from the first image with additional
    coordinates for left, right, top, and bottom boundaries to form a list of
    video coordinates. If the crop coordinates are not already set, initialize
    them to cover the entire image.

    Returns
    -------
    list of int
        A list containing the coordinates [left, right, top, bottom] for video.

    """
    if self.first_image.crop_coord is None:
        self.first_image.crop_coord = [0, self.first_image.image.shape[0], 0, self.first_image.image.shape[1]]
    if isinstance(self.top, np.ndarray):
        arenas_coord = [self.top.tolist(), self.bot.tolist(),self.left.tolist(), self.right.tolist()]
    else:
        arenas_coord = [self.top, self.bot,self.left, self.right]
    self.vars['crop_coord'] = self.first_image.crop_coord
    self.vars['arenas_coord'] = arenas_coord
    write_h5('cellects_data.h5', self.vars['crop_coord'], 'crop_coord')
    write_h5('cellects_data.h5', self.vars['arenas_coord'], 'arenas_coord')
    self.all['overwrite_unaltered_videos'] = True

save_data_to_run_cellects_quickly(new_one_if_does_not_exist=True)

Save data to a pickled file if it does not exist or update existing data.

Parameters:

Name	Type	Description	Default
new_one_if_does_not_exist	bool	Whether to create a new data file if it does not already exist. Default is True.	True

Notes

This method logs various information about its operations and handles the writing of data to a pickled file.

Source code in src/cellects/core/program_organizer.py

def save_data_to_run_cellects_quickly(self, new_one_if_does_not_exist: bool=True):
    """
    Save data to a pickled file if it does not exist or update existing data.

    Parameters
    ----------
    new_one_if_does_not_exist : bool, optional
        Whether to create a new data file if it does not already exist.
        Default is True.

    Notes
    -----
    This method logs various information about its operations and handles the writing of data to a pickled file.
    """
    data_to_run_cellects_quickly = None
    if os.path.isfile('cellects_settings.json'):
        logging.info("Update -cellects_settings.json- in the user chosen directory")
        data_to_run_cellects_quickly = read_json('cellects_settings.json')
        if data_to_run_cellects_quickly is None:
            os.remove('cellects_settings.json')
            logging.error("Failed to load cellects_settings.json before update. Remove pre existing.")
    else:
        if new_one_if_does_not_exist:
            logging.info("Create cellects_settings.json in the user chosen directory")
            data_to_run_cellects_quickly = {}
    if data_to_run_cellects_quickly is not None:
        if self.first_image is not None and self.first_image.im_combinations is not None and len(self.first_image.im_combinations) > 0:
            data_to_run_cellects_quickly['shape_number'] = self.first_image.im_combinations[self.current_combination_id]['shape_number']
        all_vars = self.all.copy()
        all_vars['vars'] = self.vars.copy()
        all_vars['vars'].pop('crop_coord', None)
        all_vars['vars'].pop('arenas_coord', None)
        all_vars['vars'].pop('exif', None)
        all_vars.pop('initial_bio_mask', None)
        all_vars.pop('initial_back_mask', None)
        data_to_run_cellects_quickly['all'] = all_vars
        write_json('cellects_settings.json', data_to_run_cellects_quickly)

save_exif()

Extract EXIF data from image or video files.

Notes

If extract_time_interval is True and unsuccessful, arbitrary time steps will be used. Timings are normalized to minutes for consistency across different files.

Source code in src/cellects/core/program_organizer.py

def save_exif(self):
    """
    Extract EXIF data from image or video files.

    Notes
    -----
    If `extract_time_interval` is True and unsuccessful, arbitrary time steps will be used.
    Timings are normalized to minutes for consistency across different files.
    """
    self.vars['time_step_is_arbitrary'] = True
    if self.all['im_or_vid'] == 1:
        if not 'dims' in self.vars:
            self.vars['dims'] = self.analysis_instance.shape[:3]
        timings = np.arange(self.vars['dims'][0])
    else:
        timings = np.arange(len(self.data_list))
        if sys.platform.startswith('win'):
            pathway = os.getcwd() + '\\'
        else:
            pathway = os.getcwd() + '/'
        if not 'extract_time_interval' in self.all:
            self.all['extract_time_interval'] = True
        if self.all['extract_time_interval']:
            self.vars['time_step'] = 1
            try:
                timings = extract_time(pathway, self.data_list, self.all['raw_images'])
                timings = timings - timings[0]
                timings = timings / 60
                time_step = np.diff(timings)
                if len(time_step) > 0:
                    time_step = np.mean(time_step)
                    digit_nb = 0
                    for i in str(time_step):
                        if i in {'.'}:
                            pass
                        elif i in {'0'}:
                            digit_nb += 1
                        else:
                            break
                    self.vars['time_step'] = round(time_step, digit_nb + 1)
                    self.vars['time_step_is_arbitrary'] = False
            except:
                pass
        else:
            timings = np.arange(0, len(self.data_list) * self.vars['time_step'], self.vars['time_step'])
            self.vars['time_step_is_arbitrary'] = False
    self.vars['exif'] = timings.tolist()
    write_h5('cellects_data.h5', self.vars['exif'], 'exif')

save_first_image()

Save the first image's validated shapes to an HDF5 file.

If the current combination ID is valid and has a non-empty set of image combinations, save the validated shapes to 'cellects_data.h5'.

Notes

This function assumes that self.first_image and its attributes are already defined. It uses the smallest memory-efficient array from np.nonzero(validated_shapes) to save space.

Source code in src/cellects/core/program_organizer.py

def save_first_image(self):
    """
    Save the first image's validated shapes to an HDF5 file.

    If the current combination ID is valid and has a non-empty set of
    image combinations, save the validated shapes to 'cellects_data.h5'.

    Notes
    -----
    This function assumes that `self.first_image` and its attributes are already defined.
    It uses the smallest memory-efficient array from `np.nonzero(validated_shapes)` to save space.
    """
    if self.first_image is not None and self.first_image.im_combinations is not None and len(self.first_image.im_combinations) > 0:
        validated_shapes = self.first_image.im_combinations[self.current_combination_id]['binary_image']
        write_h5('cellects_data.h5', smallest_memory_array(np.nonzero(validated_shapes)), 'validated_shapes')

save_masks(remove_unused_masks=True)

Conditionally save or remove masks to disk for batch processing (several folders).

When analyzing several folders, the same masks are (optionally) saved to ease the first image detection. After user input, unused masks should be removed while at other times, calling this method should not remove that information.

Parameters:

Name	Type	Description	Default
remove_unused_masks	bool	If True and there is no user-made mask, remove saved masks from disk. Default is True.	True

Notes

This function saves the masks to an HDF5 file saved in the config folder (to be accessible anywhere)

Source code in src/cellects/core/program_organizer.py

def save_masks(self, remove_unused_masks: bool = True):
    """
    Conditionally save or remove masks to disk for batch processing (several folders).

    When analyzing several folders, the same masks are (optionally) saved to ease the first image detection.
    After user input, unused masks should be removed while at other times,
    calling this method should not remove that information.


    Parameters
    ----------
    remove_unused_masks : bool, optional
        If True and there is no user-made mask, remove saved masks from disk.
        Default is True.

    Notes
    -----
    This function saves the masks to an HDF5 file saved in the config folder (to be accessible anywhere)
    """
    if self.all['keep_cell_and_back_for_all_folders']:
        if self.bio_mask is not None:
            write_h5(CONFIG_DIR / 'masks.h5', self.bio_mask, 'initial_bio_mask')
        if self.back_mask is not None:
            write_h5(CONFIG_DIR / 'masks.h5', self.back_mask, 'initial_back_mask')
        if remove_unused_masks:
            if self.back_mask is None:
                remove_h5_key(CONFIG_DIR / 'masks.h5', 'initial_back_mask')
            if self.bio_mask is None:
                remove_h5_key(CONFIG_DIR / 'masks.h5', 'initial_bio_mask')
    else:
        self.all.pop('initial_bio_mask', None)
        self.all.pop('initial_back_mask', None)
        if os.path.isfile(CONFIG_DIR / 'masks.h5'):
            os.remove(CONFIG_DIR / 'masks.h5')

save_origins_and_backgrounds_lists()

Create origins and background lists for image processing.

Extended Description

This method generates the origin and background lists by slicing the first image and its background subtraction based on predefined boundaries. It handles cases where the top, bottom, left, and right boundaries are not yet initialized.

Notes

This method directly modifies the input image data. The self.vars dictionary is populated with lists of sliced arrays from the first image and its background.

Attributes:

Name	Type	Description
self.vars	dict	Dictionary to store processed data.
self.first_image	ImageObject	The first image object containing validated shapes and background subtraction arrays.

Source code in src/cellects/core/program_organizer.py

def save_origins_and_backgrounds_lists(self):
    """
    Create origins and background lists for image processing.

    Extended Description
    --------------------
    This method generates the origin and background lists by slicing the first image
    and its background subtraction based on predefined boundaries. It handles cases where
    the top, bottom, left, and right boundaries are not yet initialized.

    Notes
    -----
    This method directly modifies the input image data. The `self.vars` dictionary is populated
    with lists of sliced arrays from the first image and its background.

    Attributes
    ----------
    self.vars : dict
        Dictionary to store processed data.
    self.first_image : ImageObject
        The first image object containing validated shapes and background subtraction arrays.
    """
    logging.info("Create origins and background lists")
    if self.top is None:
        self._whole_image_bounding_boxes()

    if not self.first_image.validated_shapes.any():
        if self.vars['convert_for_motion'] is not None:
            self.vars['convert_for_origin'] = self.vars['convert_for_motion']
        self.fast_first_image_segmentation()
    first_im = self.first_image.validated_shapes
    for rep, arena_label in enumerate(self.vars['analyzed_individuals']):
        origin_coord = np.nonzero(first_im[self.top[rep]:self.bot[rep], self.left[rep]:self.right[rep]])
        write_h5(f'ind_{arena_label}.h5', np.array(origin_coord), 'origin_coord')
    if self.vars['subtract_background']:
        if self.first_image.subtract_background is None:
            self.get_background_to_subtract()
        for rep in np.arange(len(self.vars['analyzed_individuals'])):
            background = self.first_image.subtract_background[self.top[rep]:self.bot[rep], self.left[rep]:self.right[rep]]
            write_h5(f'ind_{arena_label}.h5', background, 'background')
            if self.vars['convert_for_motion']['logical'] != 'None':
                background2 = self.first_image.subtract_background2[self.top[rep]:self.bot[rep], self.left[rep]:self.right[rep]]
                write_h5(f'ind_{arena_label}.h5', background2, 'background2')

save_tables(with_last_image=True)

Exports analysis results to CSV files and saves visualization outputs.

Generates the following output: - one_row_per_arena.csv, one_row_per_frame.csv : Tracking data per arena/frame. - cellects_settings.json : Full configuration settings for reproducibility.

Parameters:

Name	Type	Description	Default
with_last_image	bool	Also save the last image. Create duplicate when the analysis contains only one image.	True

Raises:

Type	Description
PermissionError	If any output file is already open in an external program (logged and re-raised).

Notes

Ensure no exported CSV files are open while running this method to avoid permission errors. This function will fail gracefully if the files cannot be overwritten.

Source code in src/cellects/core/program_organizer.py

def save_tables(self, with_last_image: bool=True):
    """
    Exports analysis results to CSV files and saves visualization outputs.

    Generates the following output:
    - one_row_per_arena.csv, one_row_per_frame.csv : Tracking data per arena/frame.
    - cellects_settings.json : Full configuration settings for reproducibility.

    Parameters
    ----------
    with_last_image : bool, optional
        Also save the last image. Create duplicate when the analysis contains only one image.

    Raises
    ------
    PermissionError
        If any output file is already open in an external program (logged and re-raised).

    Notes
    -----
    Ensure no exported CSV files are open while running this method to avoid permission errors. This
    function will fail gracefully if the files cannot be overwritten.

    """
    logging.info("Save results tables and validation images")
    if not self.vars['several_blob_per_arena']:
        try:
            self.one_row_per_arena.to_csv("one_row_per_arena.csv", sep=";", index=False, lineterminator='\n')
            del self.one_row_per_arena
        except PermissionError:
            logging.error("Never let one_row_per_arena.csv open when Cellects runs")
        try:
            self.one_row_per_frame.to_csv("one_row_per_frame.csv", sep=";", index=False, lineterminator='\n')
            del self.one_row_per_frame
        except PermissionError:
            logging.error("Never let one_row_per_frame.csv open when Cellects runs")
    if self.all['extension'] == '.JPG':
        extension = '.PNG'
    else:
        extension = '.JPG'
    if with_last_image:
        cv2.imwrite(f"Analysis efficiency, last image{extension}", self.last_image.bgr)
    cv2.imwrite(
        f"Analysis efficiency, {np.ceil(self.vars['img_number'] / 10).astype(np.uint64)}th image{extension}",
        self.first_image.bgr)

save_variable_dict()

Saves the configuration dictionaries (self.all and self.vars) to a json file.

If bio_mask or back_mask are not required for all folders, they are excluded from the saved data.

Notes

This method is used to preserve state between Cellects sessions or restart scenarios.

Source code in src/cellects/core/program_organizer.py

def save_variable_dict(self):
    """
    Saves the configuration dictionaries (`self.all` and `self.vars`) to a json file.

    If bio_mask or back_mask are not required for all folders, they are excluded from the saved data.

    Notes
    -----
    This method is used to preserve state between Cellects sessions or restart scenarios.
    """
    logging.info("Update -cellects_settings.json- in the Cellects folder")
    all_vars = self.all.copy()
    all_vars['vars'] = self.vars.copy()
    all_vars['vars'].pop('crop_coord', None)
    all_vars['vars'].pop('arenas_coord', None)
    all_vars['vars'].pop('exif', None)
    all_vars.pop('initial_bio_mask', None)
    all_vars.pop('initial_back_mask', None)
    write_json(ALL_VARS_JSON_FILE, all_vars)

update_available_core_nb(image_bit_number=256, video_bit_number=140)

Update available computation resources based on memory and processing constraints.

Parameters:

Name	Type	Description	Default
image_bit_number	int	Number of bits per image pixel (default is 256).	256
video_bit_number	int	Number of bits per video frame pixel (default is 140).	140

Other Parameters:

Name	Type	Description
lose_accuracy_to_save_memory	bool	Flag to reduce accuracy for memory savings.
convert_for_motion	dict	Conversion settings for motion analysis.
already_greyscale	bool	Flag indicating if the image is already greyscale.
save_coord_thickening_slimming	bool	Flag to save coordinates for thickening and slimming.
oscilacyto_analysis	bool	Flag indicating if oscilacyto analysis is enabled.
save_coord_network	bool	Flag to save coordinates for network analysis.

Returns:

Type	Description
float	Rounded absolute difference between available memory and necessary memory in GB.

Notes

Performance considerations and limitations should be noted here if applicable.

Source code in src/cellects/core/program_organizer.py

def update_available_core_nb(self, image_bit_number=256, video_bit_number=140):# video_bit_number=176
    """
    Update available computation resources based on memory and processing constraints.

    Parameters
    ----------
    image_bit_number : int, optional
        Number of bits per image pixel (default is 256).
    video_bit_number : int, optional
        Number of bits per video frame pixel (default is 140).

    Other Parameters
    ----------------
    lose_accuracy_to_save_memory : bool
        Flag to reduce accuracy for memory savings.
    convert_for_motion : dict
        Conversion settings for motion analysis.
    already_greyscale : bool
        Flag indicating if the image is already greyscale.
    save_coord_thickening_slimming : bool
        Flag to save coordinates for thickening and slimming.
    oscilacyto_analysis : bool
        Flag indicating if oscilacyto analysis is enabled.
    save_coord_network : bool
        Flag to save coordinates for network analysis.

    Returns
    -------
    float
        Rounded absolute difference between available memory and necessary memory in GB.

    Notes
    -----
    Performance considerations and limitations should be noted here if applicable.

    """
    if self.vars['lose_accuracy_to_save_memory']:
        video_bit_number -= 56
    if self.vars['convert_for_motion']['logical'] != 'None':
        video_bit_number += 64
        if self.vars['lose_accuracy_to_save_memory']:
            video_bit_number -= 56
    if self.vars['already_greyscale']:
        video_bit_number -= 64
    if self.vars['save_coord_thickening_slimming'] or self.vars['oscilacyto_analysis']:
        video_bit_number += 16
        image_bit_number += 128
    if self.vars['save_coord_network']:
        video_bit_number += 8
        image_bit_number += 64

    if isinstance(self.bot, list):
        one_image_memory = np.multiply((self.bot[0] - self.top[0]),
                                    (self.right[0] - self.left[0])).max().astype(np.uint64)
    else:
        one_image_memory = np.multiply((self.bot - self.top).astype(np.uint64),
                                    (self.right - self.left).astype(np.uint64)).max()
    one_video_memory = self.vars['img_number'] * one_image_memory
    necessary_memory = (one_image_memory * image_bit_number + one_video_memory * video_bit_number) * 1.16415e-10
    available_memory = (virtual_memory().available >> 30) - self.vars['min_ram_free']
    max_repeat_in_memory = int(available_memory // necessary_memory)
    if max_repeat_in_memory > 1:
        max_repeat_in_memory = max(int(available_memory // (2 * necessary_memory)), 1)


    self.cores = min(self.all['cores'], max_repeat_in_memory)
    if self.cores > self.sample_number:
        self.cores = self.sample_number
    return np.round(np.absolute(available_memory - necessary_memory), 3)

update_folder_id(sample_number, folder_name='')

Update the current working directory and data list based on the given sample number and optional folder name.

Parameters:

Name	Type	Description	Default
sample_number	int	The number of samples to analyze.	required
folder_name	str	The name of the folder to change to. Default is an empty string.	''

Notes

This function changes the current working directory to the specified folder name and updates the data list based on the file names in that directory. It also performs sorting of the data list and checks for strong variations in file names.

Source code in src/cellects/core/program_organizer.py

def update_folder_id(self, sample_number: int, folder_name: str=""):
    """
    Update the current working directory and data list based on the given sample number
    and optional folder name.

    Parameters
    ----------
    sample_number : int
        The number of samples to analyze.
    folder_name : str, optional
        The name of the folder to change to. Default is an empty string.

    Notes
    -----
    This function changes the current working directory to the specified folder name
    and updates the data list based on the file names in that directory. It also performs
    sorting of the data list and checks for strong variations in file names.

    """
    os.chdir(Path(self.all['global_pathway']) / folder_name)
    self.data_list = insensitive_glob(
        self.all['radical'] + '*' + self.all['extension'])  # Provides a list ordered by last modification date
    # Sorting is necessary when some modifications (like rotation) modified the last modification date
    self._sort_data_list()
    if self.all['im_or_vid'] == 1:
        self.sample_number = sample_number
    else:
        self.vars['img_number'] = len(self.data_list)
        self.sample_number = sample_number
    if not 'analyzed_individuals' in self.vars:
        self._set_analyzed_individuals()

update_one_row_per_arena(i, table_to_add)

Update one row of the dataframe per arena.

Add a row to a DataFrame for each arena, based on the provided table_to_add. If no previous rows exist, initialize a new DataFrame with zeros.

Parameters:

Name	Type	Description	Default
i	int	Index of the arena to update.	required
table_to_add	dict	Dictionary containing values to add. Keys are column names, values are the data.	required

Source code in src/cellects/core/program_organizer.py

def update_one_row_per_arena(self, i: int, table_to_add):
    """
    Update one row of the dataframe per arena.

    Add a row to a DataFrame for each arena, based on the provided table_to_add. If no previous rows exist,
    initialize a new DataFrame with zeros.

    Parameters
    ----------
    i : int
        Index of the arena to update.
    table_to_add : dict
        Dictionary containing values to add. Keys are column names, values are the data.

    """
    if not self.vars['several_blob_per_arena']:
        if self.one_row_per_arena is None:
            self.one_row_per_arena = pd.DataFrame(np.zeros((len(self.vars['analyzed_individuals']), len(table_to_add)), dtype=float),
                                        columns=table_to_add.keys())
        self.one_row_per_arena.iloc[i, :] = table_to_add.values()

update_one_row_per_frame(i, j, table_to_add)

Update a range of rows in self.one_row_per_frame DataFrame with values from table_to_add.

Parameters:

Name	Type	Description	Default
i	int	The starting row index to update in self.one_row_per_frame.	required
j	int	The ending row index (exclusive) to update in self.one_row_per_frame.	required
table_to_add	dict	A dictionary where keys are column labels and values are lists or arrays of data to insert into self.one_row_per_frame.	required

Notes

Ensures that one row per arena is being updated. If self.one_row_per_frame is None, it initializes a DataFrame to hold the data.

Source code in src/cellects/core/program_organizer.py

def update_one_row_per_frame(self, i: int, j: int, table_to_add):
    """
    Update a range of rows in `self.one_row_per_frame` DataFrame with values from
    `table_to_add`.

    Parameters
    ----------
    i : int
        The starting row index to update in `self.one_row_per_frame`.
    j : int
        The ending row index (exclusive) to update in `self.one_row_per_frame`.
    table_to_add : dict
        A dictionary where keys are column labels and values are lists or arrays of
        data to insert into `self.one_row_per_frame`.
    Notes
    -----
    Ensures that one row per arena is being updated. If `self.one_row_per_frame` is
    None, it initializes a DataFrame to hold the data.
    """
    if not self.vars['several_blob_per_arena']:
        if self.one_row_per_frame is None:
            self.one_row_per_frame = pd.DataFrame(index=range(len(self.vars['analyzed_individuals']) *
                                                    self.vars['img_number']),
                                        columns=table_to_add.keys())

        self.one_row_per_frame.iloc[i:j, :] = table_to_add

update_output_list()

Update the output list with various descriptors from the analysis results.

This method processes different types of descriptors and assigns them to the self.vars['descriptors'] dictionary. It handles special cases for descriptors related to 'xy' dimensions and ensures that all relevant metrics are stored in the output list.

Source code in src/cellects/core/program_organizer.py

def update_output_list(self):
    """
    Update the output list with various descriptors from the analysis results.

    This method processes different types of descriptors and assigns them to
    the `self.vars['descriptors']` dictionary. It handles special cases for
    descriptors related to 'xy' dimensions and ensures that all relevant metrics
    are stored in the output list.
    """
    self.vars['descriptors'] = {}
    for descriptor in self.all['descriptors'].keys():
        if descriptor == 'standard_deviation_xy':
            self.vars['descriptors']['standard_deviation_x'] = self.all['descriptors'][descriptor]
            self.vars['descriptors']['standard_deviation_y'] = self.all['descriptors'][descriptor]
        elif descriptor == 'skewness_xy':
            self.vars['descriptors']['skewness_x'] = self.all['descriptors'][descriptor]
            self.vars['descriptors']['skewness_y'] = self.all['descriptors'][descriptor]
        elif descriptor == 'kurtosis_xy':
            self.vars['descriptors']['kurtosis_x'] = self.all['descriptors'][descriptor]
            self.vars['descriptors']['kurtosis_y'] = self.all['descriptors'][descriptor]
        elif descriptor == 'major_axes_len_and_angle':
            self.vars['descriptors']['major_axis_len'] = self.all['descriptors'][descriptor]
            self.vars['descriptors']['minor_axis_len'] = self.all['descriptors'][descriptor]
            self.vars['descriptors']['axes_orientation'] = self.all['descriptors'][descriptor]
        else:
            if np.isin(descriptor, list(from_shape_descriptors_class.keys())):

                self.vars['descriptors'][descriptor] = self.all['descriptors'][descriptor]
    self.vars['descriptors']['newly_explored_area'] = self.vars['specimen_activity'] == 'move' or self.vars['specimen_activity'] == 'move and grow'

update_variable_dict()

Update the all and vars dictionaries with new data from DefaultDicts.

This method updates the all and vars dictionaries of the current object with data from a new instance of DefaultDicts. It checks if any keys or descriptors are missing and adds them accordingly.

Examples:

>>> organizer = ProgramOrganizer()
>>> organizer.update_variable_dict()

Source code in src/cellects/core/program_organizer.py

def update_variable_dict(self):
    """

    Update the `all` and `vars` dictionaries with new data from `DefaultDicts`.

    This method updates the `all` and `vars` dictionaries of the current object with
    data from a new instance of `DefaultDicts`. It checks if any keys or descriptors
    are missing and adds them accordingly.

    Examples
    --------
    >>> organizer = ProgramOrganizer()
    >>> organizer.update_variable_dict()
    """
    dd = DefaultDicts()
    all = len(dd.all) != len(self.all)
    vars = len(dd.vars) != len(self.vars)
    all_desc = not 'descriptors' in self.all or len(dd.all['descriptors']) != len(self.all['descriptors'])
    vars_desc = not 'descriptors' in self.vars or len(dd.vars['descriptors']) != len(self.vars['descriptors'])
    if all:
        for key, val in dd.all.items():
            if not key in self.all:
                self.all[key] = val
    if vars:
        for key, val in dd.vars.items():
            if not key in self.vars:
                self.vars[key] = val
    if all_desc:
        for key, val in dd.all['descriptors'].items():
            if not key in self.all['descriptors']:
                self.all['descriptors'][key] = val
    if vars_desc:
        for key, val in dd.vars['descriptors'].items():
            if not key in self.vars['descriptors']:
                self.vars['descriptors'][key] = val
    self._set_analyzed_individuals()