You are viewing limited content. For full access, please sign in.

Question

Question

Clear Forms Results

asked on May 17, 2016 Show version history

We have a Form with over 100,000 completed transactions. How do we efficiently get rid of the form results? This slows Forms to a crawl! Is there any way to do this other than selecting and deleting 50 at a time?

1 0

Answer

SELECTED ANSWER
replied on January 10, 2017 Show version history

For Forms 10.1, you just need to set the status=99 for those instances you want to delete in the [cf_bp_main_instances] table, then execute the Forms_MigrationDeleteMainInstances stored procedure.

For example, I want to delete all the completed instances for a process which process id is 94, then I executed following queries:

update [dbo].[cf_bp_main_instances] set status=99 where process_id in (select process_id from [cf_bp_processes] where bp_id=94) and status=2
exec Forms_MigrationDeleteMainInstances

You can get the process id from the URL when you open the process diagram of the process.

3 0

Replies

replied on February 8, 2017


Thank you Xiuhong for keeping this thread alive - I believe many Forms users got useful things out of it. It actually grows with LaserFiche - from a lot of coding of v 9 to two lines in 10.1. Can we make a step further? Is it possible to modify a query to meet this conditions:

·        Forms that have the status ‘Completed’ or ‘Cancelled’ can be deleted after 1 month.

·        Forms with a status of ‘In Progress’ or ‘Terminated’ can be deleted after 6 months

·        Forms with a status of ‘Suspended’ can be deleted after 1 year.

 

So DB admin will schedule a task to run weekly to clean up db automatically.

Thank you very much again,

3 0
replied on May 26, 2016

Hey Sheryl, this sounds like a feature request to be able to archive instances of a Forms process based on some user-provided criteria such as a date range. Maybe Laserfiche can provide a temporary solution in the interim. 

1 0
replied on May 30, 2016 Show version history

Hi Sheryl,

    Before Forms 10.1, if you delete an instance, the instance will only be marked as deleted in the database, all the related data won't be deleted actually. This has been improved in 10.1 to delete all the data from database when delete instance. You can use similar queries in Forms 9.2 to delete the instance as well as the data, following is the sql script you can refer to, you just need to modify the "prepare the instances ids to be deleted" part, in my sample, I delete the instance which is completed(status=2) and start date between '2016-05-30 00:00:00.000'and  '2016-05-30 7:00:00.000':

 

-- update the constraints before delete instances
IF OBJECT_ID('[dbo].[FK_form_subms_subms]') IS NOT NULL
BEGIN
	ALTER TABLE dbo.cf_form_submissions DROP CONSTRAINT FK_form_subms_subms
END

ALTER TABLE dbo.cf_form_submissions ADD CONSTRAINT
	FK_form_subms_subms FOREIGN KEY
	(
	submission_id
	) REFERENCES dbo.cf_submissions
	(
	submission_id
	) 
	ON UPDATE NO ACTION 
	ON DELETE CASCADE

IF OBJECT_ID('[dbo].[FK_data_subms_subms]') IS NOT NULL
BEGIN
	ALTER TABLE dbo.cf_data_submissions DROP CONSTRAINT FK_data_subms_subms
END

ALTER TABLE dbo.cf_data_submissions ADD CONSTRAINT
	FK_data_subms_subms FOREIGN KEY
	(
		submission_id
	) REFERENCES dbo.cf_submissions
	(
		submission_id
	)
	ON UPDATE NO ACTION
	ON DELETE CASCADE

-- create table to store the instances ids to be deleted
IF (SELECT TYPE_ID('dbo.Forms_IdList')) IS NULL
BEGIN
	CREATE TYPE [dbo].[Forms_IdList] AS TABLE (id int)
END
GO

-- create store procedure to delete instance and related data
CREATE PROCEDURE [dbo].[Forms_DeleteMainInstances] (
		@mainInstanceIds dbo.Forms_IdList READONLY -- the main instances to be deleted
	)
	AS
	BEGIN
		SET NOCOUNT ON;

		-- worke instances
		DELETE h FROM cf_bp_worker_instance_history h INNER JOIN cf_bp_worker_instances w ON h.instance_id = w.instance_id 
		WHERE w.bp_instance_id IN (SELECT * FROM @mainInstanceIds)
		
		DELETE r FROM cf_bp_worker_instnc_to_resume r INNER JOIN cf_bp_worker_instances w ON r.worker_instance_id = w.instance_id 
		WHERE w.bp_instance_id IN (SELECT * FROM @mainInstanceIds)

		DELETE p FROM cf_bp_instance_approvers_archive p INNER JOIN cf_bp_worker_instnc_to_resume_archive a ON p.resume_id = a.resume_id 
		INNER JOIN cf_bp_worker_instances w ON a.worker_instance_id = w.instance_id 
		WHERE w.bp_instance_id IN (SELECT * FROM @mainInstanceIds)
		
		DELETE a FROM cf_bp_worker_instnc_to_resume_archive a INNER JOIN cf_bp_worker_instances w ON a.worker_instance_id = w.instance_id 
		WHERE w.bp_instance_id IN (SELECT * FROM @mainInstanceIds)

		DELETE FROM cf_bp_worker_instances WHERE bp_instance_id IN (SELECT * FROM @mainInstanceIds)

		-- submissions
		SELECT submission_id INTO #submissions FROM cf_submissions WHERE bp_instance_id IN (SELECT * FROM @mainInstanceIds)
		SELECT submission_id INTO #prev_submissions FROM cf_submissions WHERE prev_submission_id IN (SELECT * FROM #submissions)
		UPDATE cf_submissions SET prev_submission_id = NULL WHERE prev_submission_id IN (SELECT * FROM #submissions)

		-- attachments
		DELETE a FROM cf_bp_attachment_data a LEFT JOIN cf_bp_data_attachment_mapping m ON a.attachment_id = m.attachment_id LEFT JOIN cf_bp_data d ON m.bp_data_id = d.bp_data_id
		WHERE a.deleted_on IN (SELECT * FROM #submissions) OR a.deleted_on IN (SELECT * FROM #prev_submissions)
		OR d.bp_data_id IN (SELECT * FROM #submissions) OR d.bp_data_id IN (SELECT * FROM #prev_submissions) 

		DELETE FROM cf_submissions WHERE submission_id IN (SELECT * FROM #submissions) OR submission_id IN (SELECT * FROM #prev_submissions)

		-- main instances
		DELETE FROM cf_bp_main_instances WHERE bp_instance_id IN (SELECT * FROM @mainInstanceIds)

	END
GO

-- prepare the instances ids to be deleted
DECLARE @ids Forms_IdList
INSERT INTO @ids 
select bp_instance_id from [cf_bp_main_instances] where status=2 and start_date>'2016-05-30 00:00:00.000'and start_date< '2016-05-30 7:00:00.000'

EXEC dbo.Forms_DeleteMainInstances @mainInstanceIds = @ids
GO

-- drop the type and stored procedure after deletion

DROP PROCEDURE [dbo].[Forms_DeleteMainInstances]
DROP TYPE [dbo].[Forms_IdList]
GO

If you are using Forms 10.0, you need to add following script to the "update constraint before delete instances" part:

IF OBJECT_ID('[dbo].[FK_submission_worker_inst_history]')IS NOT NULL
BEGIN
ALTER TABLE dbo.cf_submissions DROP CONSTRAINT FK_submission_worker_inst_history
END
ALTER TABLE dbo.cf_submissions ADD CONSTRAINT
	FK_submission_worker_inst_history FOREIGN KEY
	(
	history_id
	) REFERENCES dbo.cf_bp_worker_instance_history
	(
	history_id
	) ON UPDATE  NO ACTION 
	 ON DELETE  SET NULL 

 

1 0
replied on January 6, 2017

We would like to use this to clear out 100,000+ completed forms from our 10.1 installation.  Can this script be adapted or compatible with forms 10.1?

Deleting just 100 at a time from the user interface would take far too long.

 

Thank you.

0 0
SELECTED ANSWER
replied on January 10, 2017 Show version history

For Forms 10.1, you just need to set the status=99 for those instances you want to delete in the [cf_bp_main_instances] table, then execute the Forms_MigrationDeleteMainInstances stored procedure.

For example, I want to delete all the completed instances for a process which process id is 94, then I executed following queries:

update [dbo].[cf_bp_main_instances] set status=99 where process_id in (select process_id from [cf_bp_processes] where bp_id=94) and status=2
exec Forms_MigrationDeleteMainInstances

You can get the process id from the URL when you open the process diagram of the process.

3 0
replied on April 1, 2021

I really need to implement something like this in my 10.4 environment.  can you tell me which parts will work now?  do you have a template workflow that i could copy?  thank you Beth

0 0
replied on May 31, 2022

@████████ This stored procedure appears to have a cap on it of 1,000. Correct? So you'd have to run it however many times to delete all of the instances you have set to status=99.

0 0
replied on June 1, 2022 Show version history

Hi @████████, for performance concern, it only deletes 1000 instances with every execution since Forms 10.2. You can use following query instead if your business process has more than 1000 instances to delete(replace {bp_id} with the actual bp id you get the first number from the URL when open the process diagram which is applicable for Forms 10.2 to Forms 11 :

declare @RowCount as Int =0
update [dbo].[cf_bp_main_instances] set status=99 where process_id in (select process_id from [cf_bp_processes] where bp_id={bp_id}) and status=2
select @RowCount=count(*) FROM cf_bp_main_instances where status= 99
while @RowCount > 0
  begin
  exec Forms_MigrationDeleteMainInstances
set @RowCount = @@RowCount;
end;

What version of Forms are you using? Since Forms 10.4, there is out of box data maintenance functionality to help clean up the completed instances automatically, are you still using the SQL query to clean up the instances instead of using the out of box functionality?

1 0
replied on June 1, 2022

Version 11 update 2.

In this case the instances weren't completed. They were canceled mostly. We didn't care what the status was and just deleted every instance for that process_id. 

We had a timer event on a user task that invoked a forms process that should have been interrupting and wasn't. So we had inadvertently generated 60,000+ user tasks. Instead of stopping and deleting them page by page, the designer just deleted the forms process. Unfortunately something went wrong and the tasks were not completely cleaned up and no longer showed in the monitor tab but persisted in the Inbox. Luckily we are not live with the process yet and everything that had anything to do with that process_id could be purged.

This thread describes the same issue we were having https://answers.laserfiche.com/questions/199201/Manual-Delete-all-Tasks

We were able to get everything cleaned up with your suggestions here. We just had to run the script 60 times.

I could not find any out of the box solution that helped us get at the tasks that were not truly completed and no longer had a process tied to them.

1 0
replied on June 1, 2022

Thanks for sharing the details of your issues, we are looking into improving the clean up for such tasks in the next release.

1 0
replied on January 6, 2023

We have improved the performance of clean up tasks when delete process with Forms 11 Update 3.

You can see other changes of Forms 11 Update 3 from  https://support.laserfiche.com/kb/1014413/list-of-changes-for-laserfiche-forms-11-update-3 and get Forms 11 Update 3 from Laserfiche 11 package  https://support.laserfiche.com/kb/1014263/software-versions-and-fixes-included-in-the-laserfiche-11-download-package

1 0
replied on May 27, 2016

Great idea, Brandon! I will add it as a feature request.

0 0
replied on August 31, 2018 Show version history

So I would just like to add my thoughts here. I came upon this post because I was trying to determine how long forms instance information would be retrievable from Workflow. Every time I save a form I save the instance and submission IDs to a field. Then, with workflow, I use those IDs to retrieve some of the field data. I do this to minimize the fluff data I save to the template in the repository. Some of the field data is needed but for a limited time to just send notifications or fill out PDF forms within WF. So I was curious how much time I had to retrieve those variables before they are purged, if they are even purged at all.

 

This thread makes me want to point out to those interested in purging that they may not want to commit to a short purge time frame.

 

Anyhow, am I to believe from this thread that instance variables are saved infinitely? If not what is the built in purge cycle or process?

0 0
replied on August 31, 2018

That a real good point on needing to be cautious if you are referencing the data again at a later date.  From my last check, forms doesn't have any pre determined purge point so I would imagine it keeps the data till the end of time itself.  Probably has to do with all the reporting functions.  I've found that very few form types need retained for longer than a year.  At least in my scenarios. I've solved this by making a workflow that does it all for me.  I just choose how long I want to keep the form type and it handles it for me.  I do keep some stuff but a good deal gets purged weekly. 

 

https://answers.laserfiche.com/questions/129011/Workflow-to-act-as-forms-retention-program#129365

1 0
You are not allowed to follow up in this post.

Sign in to reply to this post.