banner
Home > Open Journal Systems

A shadow preservation framework for effective content-aware image retargeting process

Ankit Garg, Aleem Ali, Puneet Kumar

Abstract


In the discipline of image retargeting, the structure of shadows in the image is required to be preserved since it gives structural information about the objects. Many traditional image retargeting techniques do not pay attention to preserve the shadow objects present in the image. In this paper a shadow-preservation framework is proposed in which the saliency map is derived using segmentation-based gPb-owt-ucm approach to emphasize the salient regions. To assess the quality of saliency map a fixation prediction analysis is conducted on each section of the image using the eye tracker technology. Second, to pay more attention on the shadow objects a shadow map is developed after executing image pre-processing, shadow detection, region growing, and region filling operations. To rectify the issue of misclassification of shadow pixels erosion and dilation operations are performed. To obtain an efficient importance map, a gradient map is prepared using canny operator which is further combined with the saliency map and shadow map. The obtained importance map is supplied to the seam diversion based image retargeting (SDIR) technique to resize the image in horizontal and vertical directions. To justify the efficiency of the proposed framework’s the obtained results are compared with the existing state-of-the-art. By preserving shadows, the proposed framework enhances the overall visual quality and maintains the integrity of objects in the retargeted images. The framework’s effectiveness is validated through quantitative evaluations and visual comparisons with existing methods. The results demonstrate its potential for improving content-aware image retargeting applications, paving the way for more realistic and visually appealing image resizing techniques in various domains.

Keywords


content-aware; image resizing; retargeting; seam carving; saliency map; shadow map; segmentation; warping; Multi-operator

Full Text:

PDF

References


1. Vaquero D, Turk M, Pulli K, et al. A survey of image retargeting techniques. Applications of Digital Image Processing XXXIII 2010; 7798: 779814. doi: 10.1117/12.862419

2. Wei Y, Wen F, Zhu W, Sun J. Geodesic saliency using background priors. In: Fitzgibbon A, Lazebnik S, Perona P, et al. (editors). Computer Vision—ECCV 2012, Proceedings of 12th European Conference on Computer Vision; 7–13 October 2012; Florence, Italy. Springer; 2012. pp. 29–42.

3. Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998; 20(11): 1254–1259. doi: 10.1109/34.730558

4. Stentiford FWM. Attention-based image similarity measure with application to content-based information retrieval. Storage and Retrieval for Media Databases 2003; 5021: 221–232. doi: 10.1117/12.476255

5. Santella A, Agrawala M, DeCarlo D, et al. Gaze-based interaction for semi-automatic photo cropping. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; 22–27 April 2006; Montréal, Québec, Canada.

6. Gal R, Sorkine O, Cohen-Or D. Feature-aware texturing. In: Proceedings of the Eurographics Symposium on Rendering Techniques; 2006; Nicosia, Cyprus. pp. 297–303.

7. Golub E. PhotoCropr: A first step towards computer-supported automatic generation of photographically interesting cropping suggestions. Available online: https://www.yumpu.com/en/document/read/45331521/a-first-step-towards-computer-supported-automatic-generation-of- (accessed on 20 August 2023).

8. Liu F, Gleicher M. Automatic image retargeting with fisheye-view warping. In: Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology; 23–26 October 2005; Seattle, WA, USA. pp. 153–162.

9. Guo Y, Liu F, Shi J, et al. Image retargeting using mesh parametrization. IEEE Transactions on Multimedia 2009; 11(5): 856–867. doi: 10.1109/TMM.2009.2021781

10. Sachdeva S, Ali A, Khalid S. Telemedicine in healthcare system: A discussion regarding several practices. In: Choudhury T, Katal A, Um JS, et al. (editors). Telemedicine: The Computer Transformation of Healthcare. Springer Cham; 2022. pp. 295–310.

11. Sachdeva S, Ali A. Advanced approach using deep learning for healthcare data analysis in IOT system. In: Marriwala N, Tripathi CC, Jain S, et al. (editors). Emergent Converging Technologies and Biomedical Systems, Proceedings of ETBS 2021. Springer Singapore; 2022. pp. 163–172.

12. Sachdeva S, Ali A. Machine learning with digital forensics for attack classification in cloud network environment. International Journal of System Assurance Engineering and Management 2022; 13: 156–165. doi: 10.1007/s13198-021-01323-4

13. Chen J, Bai G, Liang S, Li Z. Automatic image cropping: A computational complexity study. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 27–30 June 2016; Las Vegas, NV, USA. pp. 507–515.

14. Kao Y, He R, Huang K. Automatic image cropping with aesthetic map and gradient energy map. In: Proceedings of 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 5–9 March 2017; New Orleans, LA, USA. pp. 1982–1986.

15. Guo G, Wang H, Shen C, et al. Automatic image cropping for visual aesthetic enhancement using deep neural networks and cascaded regression. IEEE Transactions on Multimedia 2018; 20(8): 2073–2085. doi: 10.1109/TMM.2018.2794262

16. Rahman Z, Pu YF, Aamir M, Ullah F. A framework for fast automatic image cropping based on deep saliency map detection and gaussian filter. International Journal of Computers and Applications 2019; 41(3): 207–217. doi: 10.1080/1206212X.2017.1422358

17. Garg A, Negi A. A survey on content aware image resizing methods. KSII Transactions on Internet and Information Systems (TIIS) 2020; 14(7): 2997–3017. doi: 10.3837/tiis.2020.07.015

18. Jiang W, Xu H, Chen G, et al. An improved edge-adaptive image scaling algorithm. In: Proceedings of 2009 IEEE 8th International Conference on ASIC; 20–23 October 2009; Changsha, China. pp. 895–897.

19. Liang Y, Su Z, Luo X. Patchwise scaling method for content-aware image resizing. Signal Processing 2012; 92(5): 1243–1257. doi: 10.1016/j.sigpro.2011.11.018

20. Pritch Y, Kav-Venaki E, Peleg S. Shift-map image editing. In: Proceedings of 2009 IEEE 12th International Conference on Computer Vision; 29 September–2 October 2009; Kyoto, Japan. pp. 151–158.

21. Avidan S, Shamir A. Seam carving for content-aware image resizing. ACM Transactions on Graphics (TOG) 2007; 26(3): 10-es. doi: 10.1145/1276377.1276390

22. Choi J, Kim C. Sparse seam-carving for structure preserving image retargeting. Journal of Signal Processing Systems 2016; 85: 275–283. doi: 10.1007/s11265-015-1084-3

23. Lin W, Zhang F, Lian R, et al. Seam carving algorithm based on saliency. In: Pan JS, Wu TY, Zhao Y, et al. (editors). Advances in Smart Vehicular Technology, Transportation, Communication and Applications, Proceedings of the First International Conference on Smart Vehicular Technology, Transportation, Communication and Applications; 6–8 November 2017; Kaohsiung, Taiwan. Springer Cham; 2017. pp. 282–291.

24. Li C, Hu R, Liang C, et al. Faster seam carving for video retargeting. In: Proceedings of 2018 25th IEEE International Conference on Image Processing (ICIP); 7–10 October 2018; Athens, Greece. pp. 823–827.

25. Patel D, Raman S. Accelerated seam carving for image retargeting. IET Image Processing 2019; 13(6): 885–895. doi: 10.1049/iet-ipr.2018.5283

26. Garg A, Negi A, Jindal P. Structure preservation of image using an efficient content-aware image retargeting technique. Signal, Image and Video Processing 2021; 15(1): 185–193. doi: 10.1007/s11760-020-01736-x

27. Garg A, Nayyar A, Singh AK. Improved seam carving for structure preservation using efficient energy function. Multimedia Tools and Applications 2022; 81(9): 12883–12924. doi: 10.1007/s11042-022-12003-1

28. Garg A, Singh AK. Analysis of seam carving technique: Limitations, improvements and possible solutions. The Visual Computer 2023; 39: 2683–2709. doi: 10.1007/s00371-022-02486-2

29. Kaufmann P, Wang O, Sorkine-Hornung A, et al. Finite element image warping. Computer Graphics Forum 2013; 32: 31–39. doi: 10.1111/cgf.12023

30. Li B, Duan LY, Lin CW, et al. Depth-preserving warping for stereo image retargeting. IEEE Transactions on Image Processing 2015; 24(9): 2811–2826. doi: 10.1109/TIP.2015.2431441

31. Del Gallego NP, Ilao J. Multiple-image super-resolution on mobile devices: An image warping approach. EURASIP Journal on Image and Video Processing 2017; 2017: 8. doi: 10.1186/s13640-016-0156-z

32. Islam MB, Wong LK, Low KL, Wong CO. Warping-based stereoscopic 3D video retargeting with depth remapping. In: Proceedings of 2019 IEEE Winter Conference on Applications of Computer Vision (WACV); 7–11 January 2019; Waikoloa, HI, USA. pp. 1655–1663.

33. Niu Y, Liu F, Li X, Gleicher M. Image resizing via non-homogeneous warping. Multimedia Tools and Applications 2012; 56(3): 485–508. doi: 10.1007/s11042-010-0613-0

34. Wang YS, Tai CL, Sorkine O, Lee TY. Optimized scale-and-stretch for image resizing. ACM Transactions on Graphics 2008; 27(5): 1–8. doi: 10.1145/1409060.1409071

35. Rubinstein M, Shamir A, Avidan S. Multi-operator media retargeting. ACM Transactions on Graphics 2009; 28(3): 1–11. doi: 10.1145/1531326.1531329

36. Su PC, Xiang ZH, Wu HW. SCAN: A multi-operator image retargeting scheme. In: Proceedings of Signal and Information Processing Association Annual Summit and Conference (APSIPA); 9–12 December 2014; Siem Reap, Cambodia. pp. 1–5.

37. Tsai WJ, Chen CF. Hybrid image retargeting. In: Proceedings of 2015 Visual Communications and Image Processing (VCIP); 13–16 December 2015; Singapore. pp. 1–4.

38. Zhu L, Chen Z, Chen X, Liao N. Saliency & structure preserving multi-operator image retargeting. In: Proceedings of 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 20–25 March 2016; Shanghai, China. pp. 1706–1710.

39. Garg A, Negi A. Structure preservation in content-aware image retargeting using multi-operator. IET Image Processing 2020; 14(13): 2965–2975. doi: 10.1049/iet-ipr.2019.1032

40. Karni Z, Freedman D, Gotsman C. Energy-based image deformation. Computer Graphics Forum 2009; 28(5): 1257–1268. doi: 10.1111/j.1467-8659.2009.01503.x

41. Chen R, Freedman D, Karni Z, et al. Content-aware image resizing by quadratic programming. In: Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops; 13–18 June 2010; San Francisco, CA, USA. pp. 1–8.

42. Shi M, Yang L, Peng G, Xu D. A content-aware image resizing method with prominent object size adjusted. In: Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology; 22–24 November 2010; Hong Kong, China. pp. 175–176.

43. Arbelaez P, Maire M, Fowlkes C, Malik J. Contour detection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2010; 33(5): 898–916. doi: 10.1109/TPAMI.2010.161

44. Available online: https://people.csail.mit.edu/mrub/retargetme/ (accessed on 20 August 2023).

45. Rubinstein M, Gutierrez D, Sorkine O, Shamir A. A comparative study of image retargeting. ACM Transactions on Graphics 2010; 29(6): 1–10. doi: 10.1145/1882261.1866186

46. Margolin R, Zelnik-Manor L, Tal A. How to evaluate foreground maps? In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition; 23–28 June 2014; Columbus, OH, USA. pp. 248–255.

47. Available online: https://www3.cs.stonybrook.edu/~minhhoai/projects/shadow.html (accessed on 20 August 2023).

48. Vicente TFY, Hou L, Yu CP, et al. Large-scale training of shadow detectors with noisily-annotated shadow examples. In: Leibe B, Matas J, Sebe N, et al. (editors). Computer Vision—ECCV 2016, Proceedings of 14th European Conference; 11–14 October 2016; Amsterdam, Netherlands. Springer Cham; 2016. pp. 816–832.

49. Abhayadev M, Santha T. Multi-operator content aware image retargeting on natural images. Journal of Scientific & Industrial Research 2019; 78: 193–198.

50. Available online: https://pixabay.com/ (accessed on 20 August 2023).

51. Garg A, Singh AK. Performance analysis of seam diversion based image retargeting technique based on edge detection operators. Multimedia Tools and Applications 2023; 82: 23207–23250. doi: 10.1007/s11042-022-14157-4

52. Garg A. Content-aware image retargeting technique and iterated function system: Frameworks, applications, and possible future advancements. Multimedia Tools and Applications 2023; 1–45. doi: 10.1007/s11042-023-16348-z




DOI: https://doi.org/10.32629/jai.v6i3.795

Refbacks

  • There are currently no refbacks.


Copyright (c) 2023 Ankit Garg, Aleem Ali, Puneet Kumar

License URL: https://creativecommons.org/licenses/by-nc/4.0